Powered by Smartsupp

Open-Source AI Models Are 'Scary' and Hard to Control, Warn Nobel Winner Geoffrey Hinton and Fei-Fei Li



By admin | Aug 12, 2026 | 5 min read


Open-Source AI Models Are 'Scary' and Hard to Control, Warn Nobel Winner Geoffrey Hinton and Fei-Fei Li

As initiatives like Pacing the Frontier look to major research labs to help steer AI development in a safer direction, open-source models have become a point of tension for the industry. Because they’re freely distributed and hard to police once released, open-weight models are difficult to constrain, which has led some labs to view them as a genuine threat. Yet at the Ai4 conference in Las Vegas last week, three of the most prominent AI researchers in the world—Nobel laureate Geoffrey Hinton, World Labs CEO and co-founder Fei-Fei Li, and Coursera co-founder Andrew Ng—weighed in on the debate. While they didn’t see eye to eye on every approach, all three made a compelling case for keeping AI open.

At the heart of their argument was a shared worry: letting a small group of major AI companies dictate the pace of progress. When just a few firms control access to a technology, as Apple and Google do with mobile operating systems, innovation tends to stall, and those platform owners end up shaping what gets built on top. Andrew Ng voiced concern that AI could fall into a similar pattern. “I don’t want there to be gatekeepers,” Ng said. “That limits how all of us can access AI.”

Companies naturally have an incentive to protect their competitive edge, including by shaping the rules that govern the industry. That could lead to a landscape where only the biggest, best-funded players—those with the resources to build the most advanced systems—end up dominating. Ng’s prescription was straightforward: keep the field crowded with multiple providers, letting models and companies compete instead of allowing a few giants to consolidate control. “If I were to try to give one prescription, it would be to promote openness,” Ng said, “because AI is amazing technology and I want it to be in everyone’s hands.”

But not everyone agreed that open-weight models would preserve that kind of competitive balance. Hinton, in particular, drew a sharp line between open-source software—which makes the underlying code available for anyone to inspect and modify—and open-weight models, which release the trained parameters of an AI system to the public. “Open source is great. You show people the code, and lots of people look at the lines of code and say, ‘Oh, there’s a bug.’ Open weights means you train a big model and then you give people the weights. That’s very different,” Hinton said. “I was against open [weights] because it makes it so easy for people to take these big foundation models, which are very expensive to train, and for much less money train them to do bad things like cyber attacks.”

Despite those reservations, Hinton conceded that open-weight models are already a permanent part of the AI landscape. “I think that battle’s been lost. We now have open-weight models, so the barrier to lots of people getting these big models, which was the cost of training foundation models, that barrier has disappeared. It’s too late.”

Still, accepting reality didn’t mean dismissing the risks. Hinton’s stance was clear: AI would keep advancing, and on balance, he saw that as a positive. He pointed to gains in productivity, education, and healthcare. “Worrying about the possible bad effects of AI and the things that intelligent beings might do when they’re smarter than us. I don’t think that’s unfair. I think it is unfair to label anybody who thinks like that as a fear-monger,” Hinton added.

Ng took a different angle. For him, the real question wasn’t whether open models carried risk, but who controlled access and who would come out on top in the market. Whoever built the cheaper model would hold the advantage, he argued. If China’s open-weight models gained traction across Asia, Africa, and other developing regions, he warned, they could shape how billions of people encountered ideas about democracy, freedom, and human rights. “One thing I hope we do is encourage American competitiveness and open-source AI. It turns out that AI is a tremendous source of soft power. You can see the way China’s model has tremendous accomplishment with Africa, for example,” Ng said. “But my worry is because of all the lobbying in the U.S. and the fear-mongering, building open-source AI in America is struggling to compete with open-weight models coming out of China, and my worry is that if China figures out a fundamentally more cost-efficient way to build AI, then things that are more cost-efficient have a fundamental business adoption advantage.”

Li pushed back on that framing. “It’s very dangerous to make this a dichotomy between complete openness all the way to complete closedness,” she said. “In complex software systems as well as scientific systems it’s much more nuanced.”

She turned to nuclear physics as an analogy: scientific papers are published openly, uranium is tightly regulated, and laboratory work sits somewhere in between. The takeaway, she explained, is that openness doesn’t have to be an all-or-nothing proposition. Different layers of the ecosystem can operate at different levels of transparency. She also pointed to collaborations between public and private institutions, like the Human Genome Project. The knowledge it produced became a platform others could build on, she said, letting pharmaceutical companies turn a profit, scientists push their research forward, and society reap the benefits. “So I think we have to use [AI] as that kind of infrastructure,” Li said. “We need some levels of openness, both in scientific discovery, in education, in global partnership, as well as lucrative business models for entrepreneurs. But we also will accept closed-source systems. This debate, especially at the sweeping level of ‘we can only tolerate one,’ is a false debate. We need to get to a level of nuance.”

Where the three did find common ground was on the need for some form of regulation to keep AI on a responsible path. “What we want to do is develop AI in a direction that helps people, and regulation will help us do that,” Hinton said. “You can’t leave it to people like Elon Musk and Mark Zuckerberg to decide how AI should be done.”




RELATED AI TOOLS CATEGORIES AND TAGS

Comments

Please log in to leave a comment.

No comments yet. Be the first to comment!