📦
Storage service
search
Ctrlk
  • Introduction
  • How-to: basic operations
    • Ingest a bag into the storage service
    • Look up an already-stored bag in the storage service
    • Look up the versions of a bag in the storage service
  • How to: advanced usage
    • Getting notifications of newly stored bags
  • How to: debugging errors
    • Where to find application logs
    • Manually marking ingests as failed
  • Reference/design decisison
    • The semantics of bags, ingests and ingest types
    • How identifiers work in the storage service
    • How files are laid out in the underlying storage
    • Compressed vs uncompressed bags, and the choice of tar.gz
  • Developer information/workflow
    • An API reference for the user-facing storage service APIs
    • Key technologies
    • Inter-app messaging with SQS and SNS
    • How requests are routed from the API to app containers
    • Repository layout
    • How Docker images are published to ECR
  • Wellcome-specific information
    • Our storage configurationchevron-right
      • Our three replicas: S3, Glacier, and Azure
      • Using multiple storage tiers for cost-efficiency (A/V, TIFFs)
      • Small fluctuations in our storage bill
      • Delete protection on the production storage service
    • Wellcome-specific debuggingchevron-right
    • Recovering files from our Azure replica
    • Awkward files and bags
    • Deleting files or bags bags from the storage service
gitbookPowered by GitBook
block-quoteOn this pagechevron-down
  1. Wellcome-specific information

Our storage configuration

Our three replicas: S3, Glacier, and Azurechevron-rightUsing multiple storage tiers for cost-efficiency (A/V, TIFFs)chevron-rightSmall fluctuations in our storage billchevron-rightDelete protection on the production storage servicechevron-right
PreviousHow Docker images are published to ECRchevron-leftNextOur three replicas: S3, Glacier, and Azurechevron-right