Our three replicas: S3, Glacier, and Azure

PreviousOur storage configuration NextUsing multiple storage tiers for cost-efficiency (A/V, TIFFs)

Last updated 1 year ago

Our three replicas: S3, Glacier, and Azure

We have three replicas for the storage service content:

1. A warm replica in S3

This is an S3 bucket in the storage AWS account, in Amazon's eu-west-1 (Ireland) region. Objects are stored in a mixture of the Standard-IA and Glacier storage classes and versioning is enabled.

This is the copy intended for day-to-day access.

Developers get access to these buckets as part of their standard AWS account permissions, but note there are specific IAM exclusions to prevent us from modifying objects in the prod bucket.

You can access these buckets using the AWS CLI or the AWS console.

Prod:
Staging:

2. A cold replica in S3

This is an S3 bucket in the storage AWS account, in Amazon's eu-west-1 (Ireland) region. Objects are stored in the Glacier Deep Archive storage classes and versioning is enabled.

This is the copy intended for disaster recovery.

Developers get access to these buckets as part of their standard AWS account permissions, but note there are specific IAM exclusions to prevent us from modifying objects in the prod bucket.

Prod:
Staging:

A cold replica in Azure

This is the copy intended for worst-case disaster recovery. It's stored in a different geographic location and service provider, to minimise the risk of a problem affecting all three copies at once.

You can only get access to these containers by asking D&T, and we don't grant access to it by default. Ideally there should be nobody who has write access to all three replica locations, to reduce the risk of somebody inadvertently deleting all three copies of an object.

PreviousOur storage configuration NextUsing multiple storage tiers for cost-efficiency (A/V, TIFFs)

Last updated 1 year ago

1. A warm replica in S3

This is an S3 bucket in the storage AWS account, in Amazon's eu-west-1 (Ireland) region. Objects are stored in a mixture of the Standard-IA and Glacier storage classes and versioning is enabled.

This is the copy intended for day-to-day access.

Developers get access to these buckets as part of their standard AWS account permissions, but note there are specific IAM exclusions to prevent us from modifying objects in the prod bucket.

You can access these buckets using the AWS CLI or the AWS console.

Prod:

Staging:

2. A cold replica in S3

This is an S3 bucket in the storage AWS account, in Amazon's eu-west-1 (Ireland) region. Objects are stored in the Glacier Deep Archive storage classes and versioning is enabled.

This is the copy intended for disaster recovery.

Developers get access to these buckets as part of their standard AWS account permissions, but note there are specific IAM exclusions to prevent us from modifying objects in the prod bucket.

Prod:

Staging:

A cold replica in Azure

This is an Azure Blob container in the D&T account, in Azure's West Europe (Netherlands) region, where blobs are stored in the Archive storage tier. The containers have a policy applied, and blobs are stored in the .

This is the copy intended for worst-case disaster recovery. It's stored in a different geographic location and service provider, to minimise the risk of a problem affecting all three copies at once.

The storage service accesses these containers using a . These are signed URIs that we keep in Secrets Manager; note that they're tied to the external IP address of the NAT Gateway in the storage account, so you can't use them locally.

Prod: , in the wecostorageprod storage account, in the rg-wcollarchive-prod resource group

Staging: , in the wecostoragestage storage account, in the rg-wcollarchive-stage resource group