OpenAI's Performance Factory puts Codex agents on ChatGPT slowdowns
A Codex-powered system ran over 1,700 automated investigations into ChatGPT performance regressions in three months, but only a fraction of its proposed fixes shipped as-is.
OpenAI published details on 19 November of the Performance Factory, an internal system that uses Codex agents to investigate and help fix ChatGPT performance regressions. It was built because, OpenAI says, traditional performance engineering could not keep pace with how fast the product changes.
The system has four parts: a task tracker with configurable workflows, a dispatch loop that assigns ready work to agents, an API that runs Codex in the cloud with access to the relevant repositories and observability tools, and event subscriptions that open a task when something like a performance alert fires. In just over three months it ran more than 1,700 automated investigations and identified over 600 root causes. A manual review of 100 of those diagnoses rated 70% as accurate. To catch the rest, a separate adversarial Codex session tries to disprove each diagnosis before it proceeds, and currently sends back about a quarter of them. Engineers still approve every fix before it deploys.
Why it matters: this is a rare, numbers-backed look at what happens when you point agents at a real production problem with guardrails attached. Of roughly 400 investigations that produced a proposed fix, only about 15 shipped as Codex-authored pull requests; another 50 informed a fix a human wrote. That is a useful data point for anyone sizing up how much of this kind of work an agent can actually take off a team’s plate today.
The caveat: these figures are OpenAI’s own account of its own system, not an independent audit, and the conversion rate from investigation to shipped fix is still narrow.