> For the complete documentation index, see [llms.txt](https://docs.wellcomecollection.org/incident-retros/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.wellcomecollection.org/incident-retros/2026-02-04_slow_web_site.md). # Incident retro - slow web site **Incident from:** 2026-02-04 **Incident until:** 2026-02-04 **Retro held:** 2026-02-05 * [Timeline](#timeline) * [Analysis of causes](#analysis-of-causes) * [Actions](#actions) ## Timeline 4 February 2026 See \#wc-platform-alerts:\ 11.50 LogicMonitor critical Alert - LMS2852 critical - WC - What's On\ 11.51 LogicMonitor critical Alert - LMS2672 critical - WC - Exhibition\ CLEARED critical - WC - What's On\ 11.52 CLEARED critical - WC - Exhibition\ 11.54 critical - WC - Exhibition\ CLEARED critical - WC - Exhibition\ 11.56 critical - WC - What's On\ 11.57 critical - WC - Exhibition\ CLEARED critical - WC - What's On\ CLEARED critical - WC - Exhibition\ critical - WC - Exhibition\ CLEARED critical - WC - Exhibition
11.57 RC Pages fed by our APIs render quickly, not so much the Prismic ones? But their status is green. 11.58 SB Yeah, I don't think it's a catalogue API issue, this page load instantly: 12.00 RC It's not slow locally.... Anything with AWS? 12:04 RC We've identified a faulty experience content deployment, Štěpán is kick starting a fresh one 12.07 RC Staging is also affected 12.09 SB Spike of requests coming from Vietnam. I temporarily extended the rate limiting rule to cover Vietnam 12.13 RC We think it's under control now but we'll have to talk about how many countries we're limiting… 12.23 SB the number of allowed requests is back to normal ## Analysis of causes What happened that we didn’t anticipate? Why didn’t our safeguards catch this? Blanket rate limit (2500 in 5 min window) ## Actions **GE (/RK supporting)** * Add Vietnam to rate limiting rules (not the Captcha) * Add staging rules (UK/US/IE only) to code, to be first in the order of rules * both [#12767](https://github.com/wellcomecollection/wellcomecollection.org/issues/12767) **JC - to think about for an OKR** * Investigate rate limiting/caching by areas of the site (works/items/images) **NP** * Add to Experience planning doc: find a way to ask Google not to index stage without impacting SEO [#9891](https://github.com/wellcomecollection/wellcomecollection.org/issues/9891) --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.wellcomecollection.org/incident-retros/2026-02-04_slow_web_site.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.