S3 Signed URLs #2 Versioning documents in amazon s3

Today I want to go over handling bucket versioning in AWS s3. I’m gonna cover uploading new versions of files, downloading specific versions and deleting versioned documents which, as it turns out, is a version in its own right.

You can download the code from Github and I’ve also recorded a video on this topic.

Enable bucket versioning

First things first, we need to enable bucket versioning which is as simple as setting the versioned property to true on the bucket created in the previous article .

new Bucket(scope, `exanubes-bucket`, {
	// ...properties
	versioned: true
});

Don’t forget to run cdk deploy to apply the changes.

Uploading new versions

Now that we have versioning enabled, we can upload new versions of files. To do that all we need to do is pass an existing S3 object key when generating a signed URL.

Since documents are already listed in the UI, I can easily provide the key of the document when requesting a presigned URL from aws and then provide it when requesting a signed URL.

async function updateDocument(data) {
	const name = data.get('name');
	const key = data.get('key');
	if (!key || !name) {
		return {
			error: 'name and key cannot be null',
			presignedUrl: null
		};
	}
	const result = await createPostUrl({
		bucket: BUCKET_NAME,
		fileName: name,
		key
	});

	return {
		presignedUrl: result
	};
}

Listing versions

Now that we have multiple versions of the same document, we need a way to list them. To do that we can use the listObjectVersions method on the S3Client class.

export async function getVersionsList(props) {
	const input = {
		Bucket: props.bucket,
		Prefix: props.key
	};

	const command = new ListObjectVersionsCommand(input);

	const response = await client.send(command);

	return z
		.array(versionResponseValidator)
		.parse(response.Versions)
		.sort((a, b) => +b.lastModified - +a.lastModified);
}

To receive a list of versions you need to provide AWS with the bucketname and object prefix. The prefix is the key of the document – not the version id. If you have a nested structure in your bucket, you can provide the prefix as path/to/s3-key.

Version response validator

In order to normalize the data a little bit, I used zod to validate and transform the response object.

export const versionResponseValidator = z
	.object({
		Key: z.string(),
		LastModified: z.date(),
		VersionId: z.string(),
		IsLatest: z.boolean()
	})
	.transform((arg, ctx) => ({
		key: arg.Key,
		lastModified: arg.LastModified,
		versionId: arg.VersionId,
		isLatest: arg.IsLatest
	}));

Downloading specific versions

Unlike a signed URL for uploading a new version, a signed URL for downloading a specific version requires a small change in the code. To download a specific version, we need to provide the version id when generating a signed URL. Everything else can stay the same though so it’s not a very big change.

export async function createDownloadUrl(props) {
	const input = {
		//...properties
		VersionId: props.version
	};

	const command = new GetObjectCommand(input);

	return getSignedUrl(client, command, { expiresIn: 60 });
}

Removing documents

One thing still missing is removing documents which could be a little bit strange because with versioning enabled, no document is still a version of that document. AWS handles this by creating Delete Markers.

Removing an object is very straightforward, all we need to do is provide the bucket name and the key of the document we want to remove and send it in a DeleteObjectCommand. AWS handles the rest, and delete marker becomes the newest version.

export async function removeObject(props) {
	const input = {
		Bucket: props.bucket,
		Key: props.key
	};

	const command = new DeleteObjectCommand(input);

	return client.send(command);
}

Including delete markers in the list

Before signing off, let’s also account for delete markers when listing versions. To do that we can use the DeleteMarkers property on the response object.

export async function getVersionsList(props) {
	//...code
	const response = await client.send(command);

	const deleteMarkers = response.DeleteMarkers?.map((marker) => ({
		...marker,
		deleted: true
	}));

	return z
		.array(versionResponseValidator)
		.parse(response.Versions?.concat(deleteMarkers ?? []))
		.sort((a, b) => +b.lastModified - +a.lastModified);
}

I’m mapping over the delete markers and adding a deleted property to them so that I can distinguish them from actual versions as they are indistinguishable from each other. Then I’m concatenating the two arrays before running them through my zod validator which also needs to be updated to include the deleted property.

Summary

In this article we went over bucket versioning, how to enable it, how to upload new versions and list them in the UI. We also covered downloading specific versions and removing documents which, as it turns out, create new versions as well.