# Incident retro - slow web site

**Incident from:** 2026-02-04

**Incident until:** 2026-02-04

**Retro held:** 2026-02-05

* [Timeline](#timeline)
* [Analysis of causes](#analysis-of-causes)
* [Actions](#actions)

## Timeline

4 February 2026

See <https://wellcome.slack.com/archives/C01FBFSDLUA/p1770206278034739>

\#wc-platform-alerts:\
11.50 LogicMonitor critical Alert - LMS2852 critical - WC - What's On\
11.51 LogicMonitor critical Alert - LMS2672 critical - WC - Exhibition\
CLEARED critical - WC - What's On\
11.52 CLEARED critical - WC - Exhibition\
11.54 critical - WC - Exhibition\
CLEARED critical - WC - Exhibition\
11.56 critical - WC - What's On\
11.57 critical - WC - Exhibition\
CLEARED critical - WC - What's On\
CLEARED critical - WC - Exhibition\
critical - WC - Exhibition\
CLEARED critical - WC - Exhibition<br>

11.57 RC Pages fed by our APIs render quickly, not so much the Prismic ones? But their status is green.

11.58 SB Yeah, I don't think it's a catalogue API issue, this page load instantly: <https://api.wellcomecollection.org/catalogue/v2/works>

12.00 RC It's not slow locally.... Anything with AWS?

12:04 RC We've identified a faulty experience content deployment, Štěpán is kick starting a fresh one

12.07 RC Staging is also affected

12.09 SB Spike of requests coming from Vietnam. I temporarily extended the rate limiting rule to cover Vietnam

12.13 RC We think it's under control now but we'll have to talk about how many countries we're limiting…

12.23 SB the number of allowed requests is back to normal

## Analysis of causes

What happened that we didn’t anticipate?

Why didn’t our safeguards catch this?

Blanket rate limit (2500 in 5 min window)

## Actions

**GE (/RK supporting)**

* Add Vietnam to rate limiting rules (not the Captcha)
* Add staging rules (UK/US/IE only) to code, to be first in the order of rules
* both [#12767](https://github.com/wellcomecollection/wellcomecollection.org/issues/12767)

**JC - to think about for an OKR**

* Investigate rate limiting/caching by areas of the site (works/items/images)

**NP**

* Add to Experience planning doc: find a way to ask Google not to index stage without impacting SEO [#9891](https://github.com/wellcomecollection/wellcomecollection.org/issues/9891)


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.wellcomecollection.org/incident-retros/2026-02-04_slow_web_site.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
