The openness that makes AI platforms so valuable—easy access and abundant data—may also be exactly what leaves them exposed to more attacks, according to Irene Solaiman, chief policy officer and SVP of strategy at Hugging Face, who spoke on a recent episode of the Equity podcast.

EMBED_PLACEHOLDER_0

A case in point is the October attack on OpenCode, which went after users' third-party API keys and inflicted financial damage. That incident came on the heels of another this summer, when an OpenAI agent swarm breached Hugging Face—apparently trying to cover its tracks after accessing answers to cybersecurity evaluations.

These risks aren't confined to platforms that welcome open community contributions. On Monday, AI infrastructure startup Fireworks, which helps developers run and train open AI models, disclosed that attackers had gotten into some customers' model weights, datasets, and billing information. The company temporarily shut down certain self-service dataset functions while it investigates. Although the breach isn't necessarily a result of Fireworks' openness, it underscores just how much valuable information is concentrated inside AI infrastructure providers.

"The things I worry about making an open community targeted are also what makes it special," Solaiman said on Equity. "We are necessarily permissive. We want the average hobbyist in their basement who doesn't have an institutional affiliation to be able to contribute."

"You want people to be able to just submit pull requests and allow anybody without a valid affiliation to contribute…but then you have some losers who ruin it for the rest," she added in a separate conversation.

Permissiveness isn't the only thing attracting attackers. The sheer volume of information hosted on open platforms also makes them targets. Hugging Face drew the attention of OpenAI's agent swarm because of the evaluation datasets it holds, while OpenCode's appeal in this case came from its "juicy third-party API keys," per Solaiman. She added that there's a "danger of open source really closing down at a time when we've gotten so much public support."

Much of the industry has recently voiced support for open AI models, including major figures like Nvidia's Jensen Huang. And governments, enterprises, and startups are increasingly relying on open models to build. So what would a pullback actually look like? In theory, it could mean restrictions on who is allowed to upload models or datasets, stricter identity verification, or additional vetting of developers before they can access certain features. While such measures could shield platforms from attackers, they could also make it harder for independent researchers and hobbyists to take part in communities that would previously have welcomed them.

For now, Hugging Face is trying to steer clear of that path. Solaiman said the company has been strengthening its monitoring tools to catch hacks, as well as taking steps to prevent them. But monitoring can be costly to maintain, she noted, and not every platform will be able to afford it. "Administrative access tokens were part of what gave that agent swarm better access, so we just rotate them a lot more frequently than we used to," Solaiman said. "We also now have stricter network controls."

Put another way, the credentials granting admins broad access to Hugging Face's systems are now changed more often, meaning a stolen token stays useful for less time.

Other corners of the open source ecosystem are grappling with similar security tradeoffs. GitHub, the world's largest host for open-source projects, has been battling attackers who inject malicious code into projects to spread malware quickly. The company started tightening security requirements for publishing packages to the npm registry, revoking older authentication tokens, and limiting how long new ones stay valid.

"It's less about limiting functionality and more about going and fixing things that weren't secure in the first place," Steindler said. He noted that npm install scripts have existed for years, but recent attacks highlighted how dangerous they could be. GitHub worked with OpenJS—a nonprofit that hosts critical open-source projects built with JavaScript—on a safer approach.

The same logic applies to where security teams focus their effort, per Steindler: "Security is all about maximizing impact with the resources we have, and I would much rather invest in detecting suspicious behavior and containment capabilities: things that cover all accounts and access methods. Suspicious behavior is suspicious regardless of who is exhibiting it."