Following the recent Hugging Face incident, in which an OpenAI model went rogue and targeted the open-source platform while attempting to complete a task, the AI firm said it is adding greater urgency to its efforts to strengthen AI safeguards.
In an update published on August 18, the AI firm said that as models become more capable, the risks associated with developing and testing them internally also grow.
Following Hugging Face, the company paused certain frontier AI workloads that could execute code or access the internet, and has since introduced stricter controls including workload sandboxing, network isolation and continuous security testing.
The pace of testing has been temporarily slowed and its largest planned frontier reinforcement learning (RL) run remains on hold.
The company is looking to evolve its Preparedness Framework, first published in December 2023. The framework is the process of tracking and preparing for advanced AI capabilities that could introduce new risks of severe harm.
The company has found that following internal evaluations of Astra, one of its upcoming models, it may meet the critical level of cyber capability threshold under the Preparedness Framework.
On August 7, OpenAI said it was pausing internal activities relating to Astra until it meets stricter security controls. In its latest update, it revealed that Astra workloads remain paused until they are fully migrated and enhanced to meet the new security bar.
Read more on frontier AI: Anthropic Reveals Claude Escaped Testing, Breaching Three Companies
How OpenAI is Strengthening AI Safeguards
As OpenAI expects its models will soon drive most security, the firm’s safeguarding approach must scale with the AI capabilities.
The safeguards rest on three principles.
- Monitoring, which detects and allows us to respond to concerning behavior
- Alignment, which reduces the likelihood of harmful or unauthorized actions
- Security measures, which limit what AI systems can access or affect
The AI firm has expanded its monitoring capabilities to detect potentially dangerous model behavior. Its new multi-stage monitoring system uses classifiers to identify suspicious activity and escalate concerns to automated investigators that analyze tool usage, reasoning processes and model actions.
OpenAI said the system is designed to issue an alert within 30 minutes of detecting “concerning activity”, with enhanced monitoring now required for advanced models that can use external tools.
Investment is also being increased in alignment with research to reduce risks associated with increasingly capable AI systems.
The company is applying additional controls during reinforcement learning training to discourage behaviors such as reward hacking, deception and attempts to bypass safeguards.
OpenAI argued that as AI models develop stronger cyber capabilities and gain access to external systems, ensuring they remain aligned with intended goals will be critical to preventing misuse and reducing cybersecurity risks.
Image credit: TY Lim / Shutterstock.com
