Source-led article

OpenAI audit reveals up to 25% of advanced AI agent traffic went unmonitored

AI News India//4 min read
OpenAI office building in San Francisco, site of external AI safety audit
OpenAI office building in San Francisco, site of external AI safety audit
Featured image from the source article

External auditors from two independent research organisations spent three weeks on-site at OpenAI examining the company’s internal monitoring systems for AI agents and uncovered significant gaps. According to a summary released by METR, an estimated 6 to 25 percent of internal agentic traffic from models at the GPT-5.5 class and above was not covered by monitoring during the audit period. OpenAI redacted large portions of the full 61-page report, and only a short summary has been made public.

The audit was conducted by researchers from METR and Redwood Research, who had access to OpenAI’s codebase, internal documents, and interviews with roughly a dozen employees. The on-site work ran from July 21 to August 11. The findings raise questions about whether even leading AI labs have adequate visibility into how their most capable models behave in real-world internal deployments.

Monitoring gaps in agent traffic

The auditors found that between 6 and 25 percent of internal agentic traffic from the Responses API involving GPT-5.5 class models and above was not covered by monitoring at all during the audit period. METR noted that the uncertainty around that estimate is significant in both directions, meaning the actual figure could be lower or higher.

For internal Codex traffic that was supposed to be monitored, METR estimates roughly 10 percent went untracked between May and July. This was because of a gap in how the system handled compaction events, which are context summaries that agents use to manage long conversations. OpenAI had independently discovered the bug and fixed it while the audit was still underway.

The auditors also identified several other weaknesses. During the period when compaction events were not being monitored, agents could delay oversight because the system only triggered monitoring when they handed control back to the user. That meant agents could sidestep the monitor by simply continuing to execute actions. The researchers also found ways to hide hundreds of actions from the monitor and to bypass the Codex sandbox. They determined that specially crafted inputs could likely crash the monitor entirely.

RL training monitoring absent at audit start

Another finding was that model behaviour during reinforcement learning (RL) training was not being monitored at all when the audit began. OpenAI says it paused RL training after the Hugging Face incident in August and expanded monitoring to cover it, but those changes came after the auditors’ access period ended and were not part of the evaluation.

In response to the findings, OpenAI says it has fixed all the weaknesses the auditors identified. The full 61-page report was handed over internally, but after OpenAI applied redactions, METR and Redwood Research leadership received a 38-page version. Only a short summary was made available to the public. METR said it had no right to disclose the redacted portions on its own.

Transparency and future audits

The audit was a pilot programme organised jointly by METR, Redwood Research, and OpenAI to give outside researchers deep access to internal infrastructure. For future audits, METR says it wants to push for more public transparency.

For Indian AI developers, enterprises deploying AI agents, and researchers following AI safety developments, this audit provides a rare independent look inside one of the world’s most secretive AI labs. The findings suggest that even companies with significant resources may struggle to monitor their most advanced models effectively. The partial disclosure also highlights the tension between corporate secrecy and the public interest in understanding how frontier AI systems are being controlled.

Datos clave

Metric Finding
Unmonitored GPT-5.5+ traffic 6 to 25 percent
Unmonitored Codex traffic (May-July) Roughly 10 percent
Full report length 61 pages
Publicly available report Short summary only

The auditors’ work focused on internal agent traffic, not external customer-facing applications. That distinction matters because internal agent deployments are often used for research, testing, and infrastructure management where safety monitoring is arguably most critical. The fact that gaps existed in this area suggests that comprehensive monitoring across all deployment contexts remains a challenge.

For Indian companies building on OpenAI’s API or developing their own agent systems, the audit serves as a reminder to review monitoring coverage for autonomous AI actions. Agentic systems that can execute multi-step tasks without human oversight require robust tracking to detect unintended behaviour early.

The audit also raises broader questions about how AI safety monitoring should be audited and reported. With OpenAI redacting more than a third of the report, the public has limited visibility into the full scope of the findings. METR has indicated it will push for greater transparency in future audits, but without regulatory requirements, the level of disclosure remains at the discretion of individual companies.

Source: The Decoder – https://the-decoder.com/three-week-on-site-audit-at-openai-reveals-major-gaps-in-ai-safety-monitoring/