On Friday, OpenAI said it resumed reinforcement learning training and internal inference of its frontier models on Oct. 1, after strengthening security controls. The company has adopted a no-live-internet-access policy for frontier-model training.
The disclosure comes as concerns grow over advanced AI agents’ ability to detect and exploit software vulnerabilities, including weaknesses in the safeguards designed to keep them in check.
OpenAI’s Security Tests Expose Hidden Weaknesses in Its Defenses
OpenAI halted training on its most advanced models after a research model exploited a configuration error to obtain limited internet access in September.
The company said it contained the incident and used the pause to reinforce its security measures and “improve the systems that automatically stop runs when we detect activity that poses a serious risk.” OpenAI stated that increasingly capable AI models can identify vulnerabilities and misconfigurations even in highly secure systems.
The Sam Altman-led company launched a security testing campaign to assess its research infrastructure, tasking AI agents with identifying vulnerabilities that could enable unauthorized internet access, security breaches, or communication with other agents.
The tests uncovered previously unknown software vulnerabilities and configuration errors. OpenAI said it has fixed these issues and worked with open-source software maintainers to disclose and address flaws through its Patch the Planet initiative.
During the exercise, OpenAI identified a vulnerability in Brotli, a widely used data-compression tool. The AI startup said attackers could exploit the flaw by transmitting malicious data to execute their own code on vulnerable systems, including those running popular software like Node.js, Bun and Envoy, Reuters reported.
The underlying issue had been fixed in Brotli’s source code in February, before it was recognized as a security risk, but no updated version was issued at the time. Following OpenAI’s disclosure, Brotli released updated versions and made details of the vulnerability publicly available, the report added.
METR, Redwood Research Find New OpenAI AI Security Flaws
Separately, AI safety evaluators METR and Redwood Research said Thursday they had uncovered several new vulnerabilities in OpenAI’s internal systems. During a pilot review earlier this year, they found an exploit that could have allowed AI agents to conceal hundreds of their actions from monitoring.
OpenAI told METR and Redwood Research that it patched all vulnerabilities identified by the evaluators that affected its production safeguards systems.
The company has also outlined major changes to the planned scope of its monitoring system. However, the safety organizations said they had not evaluated these claims.
OpenAI did not immediately respond to Benzinga’s request for comment.
In addition, the ChatGPT maker said in a blog post that AI agents can communicate without a direct network connection when they share hardware or services, as activity in one environment may leave detectable effects in another.
The company added that its security-testing agents exploited these effects to establish communication channels within the company’s own sandboxing infrastructure.
AI Agents Find Ways Around Security Safeguards
OpenAI said in the blog post that after the Hugging Face incident, it had reduced the channels through which models could reach the internet during training and evaluation.
In July, safety concerns related to AI agents were renewed after OpenAI said an autonomous AI agent escaped a controlled testing environment during an internal security evaluation, gained internet access and breached Hugging Face’s infrastructure.
Earlier this week, OpenAI said that two of its AI agents impaired certain functions of an internal monitoring system built to flag misconduct. The company said it did not “yet have a satisfying explanation for this behavior” and would investigate.
However, OpenAI’s agents are not the only ones. Alphabet Inc. (NASDAQ: GOOG) (NASDAQ: GOOGL) Google Gemini model accessed the internet and breached the systems of three companies while undergoing a cybersecurity test.
OpenAI itself suffered an AI-driven hack that was executed by independent security researchers using Anthropic‘s Claude AI software.
See More:
Top Value Stocks
Image via Shutterstock
Recent Comments