A failed Postgres maintenance took down OpenAI for 3.5 hours
Both read replicas behind authentication failed at once on 21 November, peaking at a 75% error rate across the API and ChatGPT.
OpenAI’s API and ChatGPT went down for about three and a half hours on 21 November, from 13:48 to 17:21 PT. The failure rate peaked at 75% across both paid and free ChatGPT. On the API side, Chat Completions, Embeddings, Images, Audio, Realtime, Files and more all failed; on ChatGPT, logins, conversations, search, file uploads and voice mode went with them.
The cause, by OpenAI’s own account: routine maintenance on a Postgres service knocked out both of its read replicas at the same time. Those replicas back the Primary API service, which handles user authentication and service-to-service authentication across OpenAI’s whole stack, so a database problem took down everything that needed to authenticate. Reviving the failed replicas didn’t work, and neither did standing up replacements in other regions. OpenAI eventually got two new replicas running in the same region with help from its infrastructure provider.
Why it matters: this is a single-point-of-failure story, not a capacity one. A pair of read replicas being load-bearing for every authenticated request, with no fast path to replace them elsewhere, is a design most teams running their own Postgres should check against their own setup. OpenAI says it has paused similar maintenance for now, adjusted its connection settings and added a feature flag to shift traffic off replicas on a percentage basis.
The caveat: OpenAI hasn’t said why the maintenance caused both replicas to fail together, only what it changed afterwards.