Archivematica @ Wellcome Collection
  • Introduction
  • High-level design
  • Storing born-digital files
    • Creating a transfer package
    • Upload a transfer package to S3
    • Check a package was stored successfully
    • Downloading a package from the storage service
    • Following a package in the dashboard
  • Service architecture
    • How does Archivematica work?
      • The Archivematica apps
      • Microservices, tasks and jobs
      • Gearman, ElastiCache, and the MCP server/client
    • How is our deployment unusual?
      • What are our extra services?
      • ECS containers on EC2, not Fargate
      • Why we forked Archivematica
    • How it fits into the wider platform
  • About our deployment
    • Using Wellcome catalogue identifiers
    • Different environments
    • Working storage: MySQL, Redis, and EBS
  • Administering Archivematica
    • Bootstrapping a new Archivematica stack
    • User management
      • How to add or remove users
      • Authentication with Azure AD
    • Upgrading to a new version of Archivematica
    • Running an end-to-end test
    • Clearing old transfers from the dashboard
  • Debugging Archivematica
    • Where to find application logs
    • Troubleshooting known errors
      • Timeout waiting for network interface provisioning to complete
      • 401 Unauthorized when the s3_start_transfer Lambda tries to run
      • "pull access denied" when running containers (and other ECS agent issues)
      • "Unauthorized for url" when logging in
      • "gearman.errors.ExceededConnectionAttempts: Exceeded 1 connection attempt(s)" in MCP server
      • NotADirectoryError in the Extract zipped transfer stage
    • Restarting services if a task is stuck
    • SSH into the Archivematica container hosts
Powered by GitBook
On this page
  1. Service architecture
  2. How is our deployment unusual?

ECS containers on EC2, not Fargate

PreviousWhat are our extra services?NextWhy we forked Archivematica

Last updated 2 years ago

Most of our services run as Docker containers in ECS, on Fargate.

We do package the Archivematica apps as Docker containers, but they run on a single EC2 instance with , not in Fargate.

This mimics how Archivematica expects to be run: all the services on the same server. We still package the services as Docker containers, orchestrated by ECS, so things behave similar to our other services -- but we're mounting a persistent EBS volume inside the containers to provide the shared file system.

As much as possible, these Docker images look similar to our other services, e.g. they're published by Buildkite, stored in ECR, managed by ECS.

We use EBS rather than EFS because Archivematica expects the file system to be synchronous. We did try using EFS, but we couldn't get it to behave reliably.

a shared EBS volume