> For the complete documentation index, see [llms.txt](https://docs.wellcomecollection.org/incident-retros/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.wellcomecollection.org/incident-retros/2026-02-04_slow_web_site.md). # Incident retro - slow web site **Incident from:** 2026-02-04 **Incident until:** 2026-02-04 **Retro held:** 2026-02-05 * [Timeline](#timeline) * [Analysis of causes](#analysis-of-causes) * [Actions](#actions) ## Timeline 4 February 2026 See \#wc-platform-alerts:\ 11.50 LogicMonitor critical Alert - LMS2852 critical - WC - What's On\ 11.51 LogicMonitor critical Alert - LMS2672 critical - WC - Exhibition\ CLEARED critical - WC - What's On\ 11.52 CLEARED critical - WC - Exhibition\ 11.54 critical - WC - Exhibition\ CLEARED critical - WC - Exhibition\ 11.56 critical - WC - What's On\ 11.57 critical - WC - Exhibition\ CLEARED critical - WC - What's On\ CLEARED critical - WC - Exhibition\ critical - WC - Exhibition\ CLEARED critical - WC - Exhibition
11.57 RC Pages fed by our APIs render quickly, not so much the Prismic ones? But their status is green. 11.58 SB Yeah, I don't think it's a catalogue API issue, this page load instantly: 12.00 RC It's not slow locally.... Anything with AWS? 12:04 RC We've identified a faulty experience content deployment, Štěpán is kick starting a fresh one 12.07 RC Staging is also affected 12.09 SB Spike of requests coming from Vietnam. I temporarily extended the rate limiting rule to cover Vietnam 12.13 RC We think it's under control now but we'll have to talk about how many countries we're limiting… 12.23 SB the number of allowed requests is back to normal ## Analysis of causes What happened that we didn’t anticipate? Why didn’t our safeguards catch this? Blanket rate limit (2500 in 5 min window) ## Actions **GE (/RK supporting)** * Add Vietnam to rate limiting rules (not the Captcha) * Add staging rules (UK/US/IE only) to code, to be first in the order of rules * both [#12767](https://github.com/wellcomecollection/wellcomecollection.org/issues/12767) **JC - to think about for an OKR** * Investigate rate limiting/caching by areas of the site (works/items/images) **NP** * Add to Experience planning doc: find a way to ask Google not to index stage without impacting SEO [#9891](https://github.com/wellcomecollection/wellcomecollection.org/issues/9891)