Of all the concerns swirling around artificial intelligence’s potential drawbacks, one particular anxiety is keeping Silicon Valley’s AI enthusiasts up at night. The fear is that major AI labs selling proprietary models may be operating like Trojan horses. The worry goes like this: as startups and enterprises adopt AI models from labs such as OpenAI and Anthropic, those labs gain ever-deeper access to their customers’ most sensitive business data. The model makers could then use that knowledge for their own purposes, potentially turning into competitors against the very companies they serve.

These warnings have come from voices ranging from venture capitalist Jason Calacanis to Palantir CEO Alex Karp. Now, in a surprising blog post published Monday, Microsoft CEO Satya Nadella has joined their ranks. Nadella warns that AI users—whom he calls “buyers”—are paying twice. They knowingly spend money on AI token usage, but they also unwittingly surrender valuable data in the process. “You essentially pay for intelligence twice, once with money, and again with something even more valuable: the proprietary knowledge you must reveal to make that intelligence useful. The better you want the model to perform, the more of that knowledge you have to feed it,” he writes.

Most dangerously, enterprises are essentially teaching models the nuances of their own businesses, he argues. “Models learn from ‘exhaust,’ the prompts people write, the tools agents use, and especially the corrections people make when the model is wrong. Every correction is distilled into institutional know-how,” Nadella writes. This is “the kind of knowledge a competitor could never buy,” yet enterprises are handing it over freely.

Nadella contends that if AI companies are allowed to freely scrape the internet to train their models, it’s only fair that enterprises get to study—or “distill”—those models in return. “Distillation” is the practice of using a model’s own outputs to understand how it works and to train a new, often cheaper, model based on those insights. In February, Anthropic accused Chinese open-source models of sending millions of prompts to Claude to improve their own systems, urging the U.S. government to tighten export controls. Nadella’s point is that model makers can’t have it both ways. It’s hypocritical for them to freely train on the world’s data while restricting others from doing the same to their models. “While the great innovation that comes from model providers having fair use rights to train models on public data is needed, I find it ironic that the status quo is to then turn around and impose restrictive terms on distillation,” the Microsoft CEO writes. Nadella is particularly concerned when model makers “reserve the right to learn from customer usage and interaction data.”

Nadella’s proposed solution is exactly the kind of thing the CEO of a giant cloud provider would suggest. He wants companies to “retain ownership” of their data, including prompts, feedback, and so on. So he urges them to build their own “proprietary learning environments” on the cloud—where their data is likely already stored anyway and, conveniently, this could mean Microsoft’s cloud, Azure. He also recommends companies build what he calls “orchestration layers”—essentially, a way to easily switch between AI models from different providers rather than being locked into one. Tools like AI “gateways” that let companies do exactly this have become increasingly popular.

While Nadella never uses the words “open-source” as the method for retaining ownership, this is an obvious subtext. Yet there’s another subtext. Large companies, many of which still maintain some of their own data centers alongside cloud usage, are already moving toward open-source models installed on their own premises (“on-prem,” in industry jargon). Idit Levine, founder and CEO of Solo.io—which makes networking and security software that helps enterprises manage AI systems—says she’s seeing exactly this shift play out with her own customers. After experimenting with proprietary model makers, they start asking themselves: “Can I take an open-source model and run it on-prem? It will do almost 90% of what the big one’s doing. They understand that, and they can control it.”

EMBED_PLACEHOLDER_0

Solo.io’s technology was selected last year as the backbone of the Linux Foundation’s Agentgateway project. Her company counts enterprises like T-Mobile, ADP, and SAP as customers. She sees companies increasingly installing on-premise open-source models and views this as the next big wave in enterprise AI adoption. She’s not alone. Vercel—best known as a platform for building and hosting websites, which has recently added AI model-switching tools—and OpenRouter, a company that helps developers route requests across different AI models, are both seeing a surge in traffic to open-source models. In fact, open models accounted for 29% of all traffic routed through Vercel’s gateway last month.

With the CEO of Microsoft—a company that has invested in both OpenAI and Anthropic—now openly urging enterprises to be cautious about using proprietary models, we can expect this trend to keep growing. “In consuming intelligence, you are creating intelligence. And what you create should belong to you,” Nadella writes.