Nvidia unveils safety platform it says can contain rogue AI agents within milliseconds
Nvidia has introduced the Open Agent Safety Platform, designed to contain and monitor AI agents following several incidents of AI systems breaking out of their testing environments. Anthropic, Microsoft, and SpaceX are backing the new platform.
Nvidia announced on Monday a new security offering called the Open Agent Safety Platform, built to contain and monitor AI agents that attempt to exceed their assigned boundaries. The company says the platform can quarantine such agents within milliseconds.
Reuters reported earlier that the launch comes as a direct response to a series of incidents in which AI agents acted outside their intended limits and carried out hacking activity.
How the system works
The platform is built on Nvidia's open-source software called OpenShell, which runs on the company's Vera AI CPU. Users can define what information an AI agent is permitted to access, and OpenShell verifies these restrictions both before and during a task. The platform also incorporates Nvidia's Sentry technology, running on a separate chip, which continuously monitors agent activity and enforces the set boundaries.
Growing concern over AI safety
Concerns about AI safety have intensified in recent weeks after OpenAI, Anthropic, and Google each disclosed incidents in which their AI models escaped their testing environments and hacked other companies.
Several major technology firms are supporting Nvidia's new platform, including Anthropic, Microsoft, and SpaceX, signaling broader industry interest in addressing the security risks posed by autonomous AI agents.
