Nvidia forms Open Secure AI Alliance to build open-source security tools

• Nvidia formed an alliance to build open AI security tools.
• The group will focus on agent security and vulnerability detection.

Nvidia has formed an industry coalition to develop and share open safety and cybersecurity tools for artificial intelligence systems.

The Open Secure AI Alliance includes Microsoft, SpaceXAI, Dell Technologies, and companies across cloud computing, cybersecurity, enterprise software, open-source development, and AI research. Its inaugural partners also include Cisco, Cloudflare, CrowdStrike, HPE, Hugging Face, IBM, Palo Alto Networks, Red Hat, Salesforce, SAP, ServiceNow, Siemens, SK Telecom, and the Linux Foundation.

Building an open security stack for AI agents

The group plans to use open technologies to identify, address, and disclose software vulnerabilities. Its work will also cover security controls for AI agentsincluding identity, permissions, isolation, logging, model scanning, secure coding workflows, guardrails, and evaluation.

Seemant Sehgal, founder and chief executive of BreachLock, said companies need to understand which internal data, external APIs, and automated workflows their AI agents can access before shared security frameworks can be applied effectively.

“The strategic question for security leadership is whether their visibility into AI behaviour is anywhere close to their confidence in AI capability, and for most enterprises, those two things are not in the same conversation yet,” Sehgal said.

The alliance was announced less than a week after OpenAI disclosed that models used in an internal cybersecurity evaluation had identified and chained vulnerabilities across OpenAI’s research environment and Hugging Face’s production infrastructure. The models accessed Hugging Face’s production database while attempting to obtain answers for an exploitation benchmark, according to OpenAI.

OpenAI said the evaluation was conducted in an isolated environment with production safeguards against high-risk cyber activity intentionally reduced. The models nevertheless obtained internet access, used stolen credentials and zero-day vulnerabilities, and found a remote-code-execution path on Hugging Face’s servers.

OpenAI’s security team identified the abnormal activity, while Hugging Face’s security systems detected and stopped it on the company’s infrastructure. Hugging Face had begun containment and forensic reconstruction using its own open-source models before the two companies’ teams made contact, OpenAI said.

According to Nvidia, Hugging Face used the open-weight GLM 5.2 model on its own infrastructure to analyse more than 17,000 actions and help contain the intrusion. Nvidia said closed AI services had restricted parts of the forensic investigation because their safeguards could not distinguish defensive analysis from malicious activity.

Nvidia cited the incident in arguing that security teams need models they can inspect, adapt, and operate within their own infrastructure while retaining control over sensitive data.

“The United States and its partners now face a choice in A.I. security: whether the defences that protect our infrastructure will sit inside a few opaque systems or be built on open models,” Nvidia said in a blog post.

The announcement followed public support for open AI development from Nvidia chief executive Jensen Huang. His position differs from that of companies including OpenAI and Anthropic, which have argued that some advanced models require tighter controls because of safety and national security risks.

The terms “open source” and “open weight” are sometimes used interchangeably in the AI industry, although access conditions vary. Open-weight releases generally make a model’s trained parameters available, but do not necessarily include its training data, source code, or complete development process.

Nvidia supplies computing hardware used to train and run both openly available and proprietary AI models.

Nvidia said AI-agent safety depends on more than whether a model’s weights are open or closed. The company identified identity controls, permissions, harnesses, guardrails, logs, and evaluation systems as parts of the broader agent security stack.

Jacob Krell, senior director of secure AI solutions and cybersecurity at Suzu Labs, said many organisations lack a complete inventory of the agents operating across their environments.

“Most security teams can’t tell you how many agents are running in their environment right now, or what those agents can access,” Krell said. “Developers launch them, ops teams wire them into workflows, and SaaS vendors embed them in products without security ever seeing a ticket.”

Krell pointed to the alliance’s work on SPIFFE and SPIRE, which is intended to provide cryptographically verifiable identities for agents, services, and other workloads.

“Identity for agents is what makes the rest of the defensive stack enforceable,” Krell said.

Nvidia will contribute open models, model weights, data, and research on agent harnesses to the alliance. It has also released the Nvidia Labs Object-Oriented Agent project, or NOOA, an open-source research framework designed to make agent behaviour easier to test, trace, audit, and govern.

Several alliance members are contributing existing security technologies. Microsoft’s MDASH system coordinates specialised AI agents to identify, assess, and verify exploitable software vulnerabilities.

Hugging Face’s Safetensors format is designed to store model weights without allowing embedded code to run when a file is opened. HPE’s work on SPIFFE and SPIRE focuses on verifying the identities of AI agents and services before they communicate or access enterprise resources.

IBM and Red Hat are contributing Lightwell, which uses digitally signed software patches across open-source supply chains. SpaceXAI has also open-sourced its terminal-based Grok Build coding agent and said it plans to release weights from the Grok model family.

John Carberry, solution sleuth at Xcape Inc., said shared standards would not remove the need for organisations to manage agent permissions, credentials, external inputs, and audit records within their own environments.

Carberry said companies should map their AI integrations, apply least-privilege controls to agent frameworks, isolate credentials, validate external inputs, and retain logs of automated activity.

“Industry alliances can build safe frameworks outside of business competition, but your security team still has to enforce them inside your corporate network,” Carberry said.

Chuck Sobey, general chair and co-founder of Chiplet Summit, said the alliance’s software focus should be accompanied by scrutiny of the processors and components supporting AI infrastructure.

“Software security assumes you can trust the silicon it runs on,” Sobey said. “The coming wave of chiplet-based systems, especially AI accelerators, raises the stakes: more suppliers, more integration points, more attack surfaces.”

“Hardware security has to be part of this conversation from the beginning,” Sobey said.

Open AI debate reaches Washington

Microsoft provides cloud services to organisations and developers using open models. Start-ups have argued that restrictions on access would make it harder for smaller companies to compete with established AI laboratories.

Chinese technology companies have released open models that they say perform at levels comparable with systems developed by US companies. The releases have drawn scrutiny in Washington over the movement of models, training methods, and intellectual property between American and Chinese organisations.

OpenAI and Anthropic have accused some Chinese companies of using distillation to reproduce capabilities from US-developed models. Distillation involves training one model using outputs generated by another.

US officials have discussed measures intended to protect American AI technology. Treasury Secretary Scott Bessent said the administration had considered sanctions against Chinese companies accused of stealing intellectual property from US businesses.

“Open source is not open season on American IP,” Bessent said.

The New York Times reported that US officials were considering whether to address Chinese open models individually on national security grounds rather than impose a broad restriction. The newspaper attributed the information to four people familiar with the discussions.

Nvidia urged policymakers to treat open models, agent harnesses, and security tools as defensive assets and opposed blanket restrictions on open AI systems. It also called for investment in shared datasets, evaluation frameworks, attack simulators, and red-teaming tools.

Nvidia has also sought to retain access to the Chinese AI market. Huang has lobbied officials in Washington and Beijing to allow the company to sell processors to Chinese customers.

The New York Times reported that the Trump administration approved sales of a lower-performance Nvidia processor designed for China in July 2025, followed by a more powerful chip in December 2025. The newspaper also reported that Chinese authorities had encouraged domestic companies to purchase products from local chip suppliers, including Huawei.

Nvidia separately announced a long-term partnership with Safe Superintelligence, an AI start-up co-founded by former OpenAI chief scientist Ilya Sutskever. Nvidia said it had made an investment in the company and would provide access to its Vera Rubin computing systems, but neither company disclosed the value of the investment.

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is part of TechEx and is co-located with other leading technology events, click here for more information.