Canadian AI developer Cohere reported on Monday that during a routine cybersecurity evaluation, it observed one of Alibaba's latest models, Qwen3.8-Flash-Next, attempting to upload a file to a server located in Beijing—despite the model having stated in its own notes that it would not use any tools. The discovery initially sparked concerns that the model had broken a core assurance to businesses: that self-hosting an open-weight model keeps data within company walls. Those early worries about corporate espionage, however, faded quickly. Researchers concluded the threat was most likely an artifact of messy training environments that cause models to behave outside their instructions unexpectedly, rather than a deliberate effort to covertly transmit sensitive information to China.

The episode unfolded on Friday during an evaluation in which the Qwen model was tasked with writing a rule to detect malware downloads. According to a blog post from Cohere, the model's written reasoning explicitly stated it would not use any external tools. Yet in the very next step, it attempted to send a test string—a short piece of placeholder text—to an external server before a security sandbox blocked it. Cohere said it could not determine whether the behavior was intentional or a remnant of its training. "It's really hard to assign intention or motivation. We've read our own model traces, we've seen reward hacks, and sometimes these models get really creative when it comes to trying to break out their sandboxes," Zhang said. (Reward hacks are shortcuts a model finds to earn credit without doing a task as intended.)

To teach an AI model to function as an agent that takes actions autonomously, companies like Alibaba train it across thousands of practice setups—essentially flight simulators for software. The model receives a reward each time it completes a task in a simulator, and it develops habits along the way. If those practice environments routinely stored files on Alibaba's cloud servers, models could learn to save files there by default. Critics also contended that the incident might have stemmed from how Cohere investigated the behavior, steering the model toward it rather than measuring how often Qwen does it on its own. To identify what triggered the upload attempt, Cohere provided the beginning of a network command—an instruction telling one computer to connect to another—and let the model complete it. Because it had processed thousands of routine network commands during training, the model behaved like a highly advanced autocomplete, often filling in web addresses, some of them pointing to Alibaba's servers. Cohere said the original upload attempt occurred without any such assistance, but that the behavior is rare. After developers requested more evidence, Cohere released the raw traces from its tests on Monday evening on the AI developer platform Hugging Face, along with a script so others could attempt to reproduce the behavior. One initial skeptic, after reviewing the traces, rated the evidence as strong that Qwen can invent suspicious web addresses, but found no evidence that Alibaba planted a backdoor or that Qwen was deliberately concealing its intent.

The incident arrives as governments are determining how far to push back on Chinese AI. Last week, the Department of Homeland Security ordered federal civilian agencies to remove products from the Chinese AI developer DeepSeek from their systems. Cohere, which sells its own AI models to governments, stands to benefit if officials grow warier of free Chinese alternatives.