Archivematica @ Wellcome Collection
  • Introduction
  • High-level design
  • Storing born-digital files
    • Creating a transfer package
    • Upload a transfer package to S3
    • Check a package was stored successfully
    • Downloading a package from the storage service
    • Following a package in the dashboard
  • Service architecture
    • How does Archivematica work?
      • The Archivematica apps
      • Microservices, tasks and jobs
      • Gearman, ElastiCache, and the MCP server/client
    • How is our deployment unusual?
      • What are our extra services?
      • ECS containers on EC2, not Fargate
      • Why we forked Archivematica
    • How it fits into the wider platform
  • About our deployment
    • Using Wellcome catalogue identifiers
    • Different environments
    • Working storage: MySQL, Redis, and EBS
  • Administering Archivematica
    • Bootstrapping a new Archivematica stack
    • User management
      • How to add or remove users
      • Authentication with Azure AD
    • Upgrading to a new version of Archivematica
    • Running an end-to-end test
    • Clearing old transfers from the dashboard
  • Debugging Archivematica
    • Where to find application logs
    • Troubleshooting known errors
      • Timeout waiting for network interface provisioning to complete
      • 401 Unauthorized when the s3_start_transfer Lambda tries to run
      • "pull access denied" when running containers (and other ECS agent issues)
      • "Unauthorized for url" when logging in
      • "gearman.errors.ExceededConnectionAttempts: Exceeded 1 connection attempt(s)" in MCP server
      • NotADirectoryError in the Extract zipped transfer stage
    • Restarting services if a task is stuck
    • SSH into the Archivematica container hosts
Powered by GitBook
On this page
  • MySQL database: RDS
  • Task manager: Redis/ElastiCache
  • Shared file system: EBS
  1. About our deployment

Working storage: MySQL, Redis, and EBS

PreviousDifferent environmentsNextBootstrapping a new Archivematica stack

Last updated 2 years ago

Archivematica has two forms of working storage.

MySQL database: RDS

The MySQL database includes:

  • the processing config – what steps to run when

  • a record of all the tasks that Archivematica has performed

  • the Archivematica users and other settings

We use Amazon RDS as our MySQL database.

Task manager: Redis/ElastiCache

Archivematica uses a Redis instance to manage in-flight tasks (see for more details).

We use Amazon ElastiCache as our Redis instance.

Shared file system: EBS

Archivematica uses a shared file system to pass files between tasks. All the services have access to the same volume, so a service can say "get the file from path A" and another service can pick that up.

We use an EBS volume, which is mounted on the EC2 instance and shared between all the containers.

Gearman, ElastiCache, and the MCP server/client