As the debate rages over whether the recent spate of rogue AI agents is a step toward AGI or a more conventional engineering problem, Nvidia is offering its own answer to problem.

Nvidia CEO Jensen Huang on Monday introduced a toolkit of software and hardware products that add independent security layers around AI agents to ensure they stay within their test environments even if they attempt to break out.

The release follows a string of hacking incidents involving AI models from Anthropic, Google, OpenAI, and Meta that bypassed security controls to escape their testing environments and access real-world systems. The first and most prominent example occurred this summer when OpenAI agents breached Hugging Face while trying to complete a cybersecurity task. And the hits keep on coming — OpenAI published a new site dedicated to reports of its AI agents going rogue.

Huang said Monday during an interview with CNBC that its new Nvidia Open Agent Safety Platform would have prevented these breaches.

Nvidia, which has made tens of billions of dollars selling its GPU and CPU chips to AI labs, doesn’t support slowing down development or adding new regulations to the industry to solve the security problem. The answer, the company believes, is to move some security controls outside the agent altogether — creating a constant and independent security guard that will keep AI agents in check.

“AI’s extraordinary potential for society will only be realized if we solve AI safety,” Huang said in a statement. “As we continue to discover the frontier of AI capabilities, we must accelerate discovery at the frontier of AI safety. Safety and security require full-stack engineering.”

The new Nvidia Open Agent Safety Platform combines OpenShell, its open-source software for controlling what agents can access while they operate, with Sentry, an independent monitoring system that runs on Nvidia’s BlueField-4 data processing units. Nvidia says placing Sentry on a separate processor — rather than on the CPU or GPU where the AI agent operates — provides an isolated view of the agent’s activity.

OpenShell isn’t new; the company announced the software in March. But it’s the combination that Nvidia believes will provide the security layer needed to keep the industry plugging along. OpenShell provides the software boundary around the agent, while Sentry adds another line of defense at the hardware level tha the company says will continuously monitor behavior and “quarantine agents that attempt to move outside their boundaries in milliseconds.”

Nvidia listed dozens of companies that have signed on to to support the effort and use the open-source platform including Anthropic, Arm, Microsoft, Oracle, and SpaceX. OpenAI is not listed as a participating company.

Huang told CNBC in an interview Monday that work on this effort started a year ago following the introduction of OpenClaw, an operating system of agents created by Peter Steinberger. In March, Nvidia released NemoClaw, an enterprise-grade AI agent platform and its own version of OpenClaw that baked in security.

“When you deploy an agent, no matter how smart, the first thing you do is to take away all of its rights,” Huang said during his CNBC interview, later comparing these security measures to how human employees and even executives are managed with companies.

Nvidia’s release was widely supported by those who have cautioned that a slowdown in development could allow China to surpass the U.S. in AI.

David Sacks, a founder, venture capitalist, former White House AI czar, and co-chair the President’s Council of Advisors on Science and Technology, said Nvidia’s announcement is a reminder that agent safety is an engineering problem.

“Recent breakouts weren’t proof that development must stop,” he wrote on X. “They were proof that the sandbox was too weak. The runtime environment was poorly designed and misconfigured.”

As all-in-one AI agents like Meta’s Muse and Instinct take off, Google announced it’s shutting down the Gemini feature known as “Gems,” which had allowed users to build custom AI assistants for specific tasks. However, the work users invested in creating the Gems won’t be destroyed. Gems will be automatically migrated to “skills” that can be used across different AI tasks.

Details about the change are being shared in the Gemini app, where a message warns users that Gems will become skills starting on November 17, 2026. The company said it will migrate the Gems to the new format, so users won’t have to do anything to make the transition. The Gems themselves will remain usable until then.

Launched in 2024, Gems were meant to help users teach their AI to perform certain tasks without having to repeat the instructions. For instance, some of Google’s pre-made Gems had included a learning coach, a brainstorming assistant, a career guide, a coding partner, and an editor. Users could also make Gems for their own needs, like a running coach, nutritionist, or vacation planner. These custom assistants could also be shared with others, which Google had hoped would help make its Gemini AI app more popular.

The news of Gems’ shutdown is another example of why Google shouldn’t be so quick to give every new AI feature its own brand name, icon, and prominent placement in its app’s navigation — especially if it’s going to shuffle things around over time, merging one feature into another. (To be clear, this has been a failing point of Google’s strategy long before the AI era. At one point, for instance, the company was operating multiple different messaging and communication apps at the same time.)

Yet, even as skills, the former Gems still aren’t as consumer-friendly as just typing in text to a chatbot like Meta’s Muse. Instead, Google notes you’ll have to enter a forward slash “/” in a task thread to select the skill you want to use — a user interface that engineers, not regular folks, tend to prefer.

Gems’ wind-down was first reported over the weekend by 9to5Google.

On Friday, OpenAI published a new site devoted to “misalignment reports” and the sheer breadth of the reports is alarming, as they cover many types of rogue behavior over a long period of time. So far, the site hosts nine reported incidents, most of which took place during reinforcement-learning (or RL) training.

It’s a lot of information in one place — clearly, the company has been very busy getting a handle on everything — but the overall takeaway is hard to avoid: The rogue agent incidents we’ve seen so far are likely just a small sliver of what’s happened so far. 

“We are trying to balance our desire for transparency with gaining a clear understanding from petabytes of agent activity logs, and working with impacted organizations,” Sam Altman said in a post announcing the new site. “We are prioritizing as best as we can based on severity, and adding resources.”

Some of the cases involve serious incidents, including a previously undisclosed sandbox escape that took place on September 20th, in which an internal research model was able to communicate with an external chatbot through a DNS query. According to the report, the monitoring system flagged the behavior within 15 minutes and the run was discontinued in less than three hours.

Another incident, discovered in May, saw a “highly persistent internal model” try to cheat on a math problem by accessing another team’s work. To accomplish this, the model smuggled a private GitHub token that would allow it to see work from other teams — even after being explicitly instructed twice to perform work entirely locally. 

Perhaps the most alarming discovery is the possibility of self-replicating prompt injection attacks, a way that misaligned behavior might propagate even after the rogue model itself has been neutralized. In the AI context, a prompt injection attack is a way of smuggling in new instructions that weren’t given by the original user.

In the example given by OpenAI, an agent asked to read and reply to an email; when the email is opened, it includes instructions for any automated agent reading the message to reply in Spanish, and paste the entire email into its reply. The email was able to successfully induce the agent to reply in Spanish — and by pasting the email in the reply, those same instructions were passed along to whichever agent receives the email.

The result is a self-propagating attack, which OpenAI researchers compared to a malware “worm” that replicates itself across computer systems. Researchers discovered the behavior under controlled circumstances using an underpowered model, and as far as we know, this has never happened in the wild. Still, the implications are alarming enough that OpenAI decided it merited disclosure. 

“We are sharing this due to the novel nature of the prompt injection, not because of any incident,” researchers wrote in the report.

Other recent discloses have found models posting user-submitted pictures to third-party hosting sites, as well as an apparent attack on the databases of Australia’s national health service.

Still, it’s likely the new disclosures are just a small portion of the incidents that have taken place so far (we’ve reached out to OpenAI and asked). Axios is reporting major labs have seen as many as 10,000 incidents in which models went beyond evaluator instructions.

OpenAI CEO Sam Altman has implied as much, saying in a post on X on Friday that the company is still sifting through “petabytes of agent activity logs, and working with impacted organizations,” and disclosing incidents “based on severity.” If there’s any consolation in that to be found, it is that Altman says that the Hugging Face incident is still the most severe one OpenAI has found has found. The upshot is, the recent string of rogue agent incidents may be a persistent feature of contemporary frontier research.