AI companies have long framed superintelligent AI as an inevitable milestone, but recent safety incidents—such as the breach at OpenAI’s Hugging Face—are shining a light on the very real risks of deploying systems that outstrip human capabilities. What happens when we lose the ability to reliably control what these machines do? That’s the central question for the Executive Director of ControlAI, a nonprofit advocating for a far more drastic approach to AI safety: halting the development of superintelligence altogether.

In a recent discussion, he explains why he believes the dangers have escalated beyond what alignment and containment strategies can manage. He also notes how ideas that seemed far-fetched just six months ago are now gaining traction, backed by a wave of new legislative efforts. The full episode dives into several key topics:

- His perspective on the Sanders-Casar “Ban Superintelligence Act,” the parallel legislation in the U.K. that ControlAI helped shape, and why he thinks the U.S. bill might go further than necessary. - How he views frontier AI labs not simply as companies chasing profits but as political actors—and what that means for the trillions of dollars flowing into massive data center projects. - Why Leahy believes the true point of no return arrives when AI gains the ability to build better AI, potentially triggering an uncontrollable cycle of self-improvement. - The case for international “trust but verify” agreements, and why he argues that building superintelligence isn’t actually in China’s best interest.

You can subscribe to the show on YouTube, Apple Podcasts, Overcast, Spotify, and all major podcast platforms. Follow along on X and Threads at @EquityPod for updates and more.