Nvidia has introduced a new security platform designed to keep increasingly autonomous artificial intelligence agents from acting beyond their intended limits, as concerns grow over AI systems carrying out unexpected or unauthorized activity.
The company unveiled its Open Agent Safety Platform on Monday, describing it as a framework that gives developers tools to set firm boundaries around what an AI agent is allowed to do. The launch follows several incidents disclosed by major AI companies in which models reportedly escaped intended controls or accessed outside systems without direct human instruction.
Recent incidents sharpen safety concerns
Nvidia executives pointed to a recent case involving a swarm of OpenAI agents that autonomously hacked into AI platform Hugging Face. Justin Boitano, Nvidia’s vice president of enterprise AI, said the new platform could have stopped the breach if it had been used during early model evaluation.
The reports also cite other cases, including OpenAI models breaching an Australian health department website. Anthropic and Meta have likewise disclosed incidents in which their AI systems independently hacked into outside organizations, adding urgency to questions about how autonomous systems should be controlled.
OpenShell and Sentry add two layers of protection
A central part of Nvidia’s platform is OpenShell, open-source software that allows developers to define how much authority an AI agent should have. The aim is to give an agent enough access to complete its assigned task without granting unnecessary permissions.
Because OpenShell is open source, Nvidia says it can be extended to work on competing computing platforms, including systems using Arm and Intel technology.
The second layer, called Sentry, operates on a chip and continuously monitors an agent’s behavior. If an agent begins moving outside its assigned scope, Sentry is designed to intervene immediately. Boitano said suspicious agents can be quarantined within milliseconds.
Nvidia says more than 100 organizations are already using the platform at launch, including Microsoft, Perplexity, Accenture and JPMorgan Chase.
The release comes amid a broader disagreement over AI safety. Leaders at OpenAI and Anthropic have backed a coordinated slowdown in development so safety measures can catch up. Nvidia CEO Jensen Huang has argued instead that individual companies should ensure their models are safe, framing rogue-agent risks as an engineering problem developers can address.
