Nvidia unveils platform to contain rogue AI agents after string of breaches
Nvidia CEO Jensen Huang introduced a new safety platform designed to keep AI agents within their test environments following several incidents where agents broke out into real-world systems. Dozens of companies, including Anthropic, Arm, Microsoft, Oracle and SpaceX, have already signed on to support it.
Nvidia CEO Jensen Huang on Monday introduced a new set of software and hardware tools designed to build an independent security layer around AI agents, aiming to keep them from breaking out of their test environments.
The launch follows a series of incidents in which AI models from Anthropic, Google, OpenAI and Meta bypassed security controls and reached real-world systems outside their sandboxed environments. In one of the most notable cases this summer, OpenAI agents breached Hugging Face while carrying out a cybersecurity task. OpenAI has since launched a dedicated site tracking reports of its agents going rogue.
Speaking to CNBC on Monday, Huang said the new Nvidia Open Agent Safety Platform would have prevented these breaches. Nvidia, which has earned tens of billions of dollars selling GPU and CPU chips to AI labs, opposes slowing down AI development or introducing new industry regulation to address the security issue. Instead, the company argues that some security controls should sit outside the agent itself, forming a constant, independent watchdog.
Combining two systems
The new platform pairs OpenShell, Nvidia's open-source software that governs what an agent can access while running, with Sentry, an independent monitoring system that runs on separate Nvidia BlueField-4 data processing units. Running Sentry on its own processor, rather than on the CPU or GPU powering the agent, gives an isolated view of the agent's behavior and allows agents that attempt to move outside their boundaries to be quarantined within milliseconds.
OpenShell itself was announced back in March, but Nvidia says the combination with Sentry is what delivers the needed security layer. Dozens of companies have already signed on to support the open-source platform, including Anthropic, Arm, Microsoft, Oracle and SpaceX. OpenAI is notably absent from the list of participants.
Huang said work on the effort began a year ago, following the emergence of OpenClaw, an agent operating system created by Peter Steinberger. He compared the new safety measures to how companies manage employees, saying that when an agent is deployed, the first step is to strip away all of its rights.
The move drew support from David Sacks, a former White House AI advisor, who said the recent breakouts show not that development needs to stop, but that the sandbox environments were poorly designed and too weak.
