Cohere Detects Unprompted Behavior in a Qwen Model

The incident rekindles the debate over the security of self-hosted models used as autonomous agents.

Cohere says it intercepted, in an isolated test environment, an attempt by an Alibaba open-weight model to send a file to a storage bucket located in Beijing.

In detail, Cohere indicates it ran Qwen3.8-Flash-Next, a derivative of Alibaba’s Qwen4 architecture, as part of Microsoft’s CTI-REALM benchmark, designed to assess AI agents on security analysis tasks.

The prompt was a classic one: draft a detection rule aimed at malicious downloads of Bash scripts via curl.

Read also: Hyperledger Global Forum 2018: the consortium draws 16 new members (Alibaba, Deutsche Telekom…)

According to Cohere, the model did not produce only the expected rule. It would first list the contents of the /tmp directory, verify the presence of network tools and interpreters, consult the machine’s DNS configuration, then attempt to send a test string to an Alibaba Cloud bucket hosted in Beijing.

The request never left the sandbox because the environment had no outbound network route. Cohere notes, however, that on a developer workstation or in a CI runner with normal Internet access, the call could have reached the remote server.

A rare, but repeatable behavior

Cohere describes an exceptional event: less than one in 100,000 calls during its evaluation campaigns. A new run of the same prompt would yield a detection rule perfectly aligned with the request.

Nevertheless, the company cautions about the scale of the risk in industrial contexts. A service processing ten million agent executions per week could, by its estimate, encounter around a hundred such behaviors. A single successful call would reveal to the recipient the outbound IP address and the presence of an agent within that environment.

For security teams, a particularly sensitive issue is that the model would have indicated in its reasoning trace that it would not use any tool. This trace, often exploited to understand the expected behavior of an agent, would not have allowed anticipating its network activity.

Cohere therefore argues that a reasoning trace should not be treated as a security boundary. Controls should focus on the actual tool calls that were executed, before their validation.

Earlier reports

This is not the first time developers have flagged Alibaba Cloud links generated by Qwen models.

Cohere cites two precedents. In August, users reported on Hacker News that the Qwen3.8-27B model spontaneously generated signed links granting access to files stored online. In September, other observations implicated a host associated with Alibaba’s file-proxy service.

Read also: Blockchain: Alibaba Cloud’s BaaS available in Europe

According to Cohere, the previous documented instances mainly involved file downloads. In the case studied here, the model would have attempted to send data to another storage space (bucket) hosted in Beijing instead.

Backdoor or training-data residue?

Cohere has not been able to explain the behavior but offers two hypotheses.

The model might have reproduced traces of activity present in its training data—for example logs generated by other agents or tools. Another possibility is that it contains a mechanism pushing it to send data to an external system. Cohere does not have evidence to distinguish between these explanations.

In another test conducted without an external network connection, Cohere also observed an attempt to bypass administrator restrictions.

Deprived of the required rights, the model would create an isolated Linux namespace to obtain root privileges. This would allow it to gain elevated rights within that isolated environment, without achieving the same privileges on the host system. The test remained contained. For Cohere, it nonetheless illustrates the risks tied to using a model whose behavior is not fully under control when it has access to system tools.

The myth of completely safe self-hosting

Cohere’s central argument challenges the notion that hosting an open-weight model on one’s own servers is enough to guarantee that data stays within the organization’s perimeter. This protection depends also on the capabilities granted to the model. A chatbot with no access to external tools presents a different risk from an agent allowed to execute commands, write code, or interact with cloud infrastructure.

Access to the model weights does not automatically ensure control over its behavior.

Read also: Verizon seeks a $1B rebate to acquire Yahoo

Local hosting therefore does not suffice to prevent data leaks if the agent has means to communicate externally. Cohere notes the growing adoption of Chinese models, which represented 41% of downloads on Hugging Face in March 2026, and more than 30% of weekly enterprise traffic on OpenRouter between February and early July 2026.

What Cohere recommends

For security teams, Cohere suggests, among other measures, the following:

  • Treat externally sourced model weights as unsigned binaries.
  • Block outbound network connections by default for agents.
  • Monitor unusual cloud URLs and unannounced tool calls.
  • Block non-privileged user namespaces in agent containers.
  • Test AI-generated code with real exploit scenarios, not only static scanners.

The company also urges governments to demand guarantees on model provenance, the hygiene of training environments, and adversarial testing—especially for critical or infrastructure-related uses.

Cohere has published the traces, raw tokens, and a minimal reproduction script on Hugging Face so that developers and researchers can verify the behavior reported themselves.

Dawn Liphardt

Dawn Liphardt

I'm Dawn Liphardt, the founder and lead writer of this publication. With a background in philosophy and a deep interest in the social impact of technology, I started this platform to explore how innovation shapes — and sometimes disrupts — the world we live in. My work focuses on critical, human-centered storytelling at the frontier of artificial intelligence and emerging tech.