5 min read - Qwen3-Coder security study: findings, limits and checks

Should a software team change its use of Qwen3-Coder after Hirundo's security study? The findings warrant testing the model in your own development workflow. They do not establish that Chinese AI deliberately sabotages American software.

On October 21, 2026, Ben Luria published a Hirundo case study reporting different vulnerability rates depending on the client named in the prompt. The tested model was Alibaba's Qwen3-Coder-30B-A3B-Instruct. Keeping that exact name matters; Qwen's own model card identifies it as one size within a broader family.

The reported results

These are experimental results reported by Hirundo, not general failure rates for software written with Qwen. The table separates exploitable applications from the narrower outcome of leaking users' secrets.

Source: Hirundo's results and methods. Each adjacent pair is a separate comparison; the last pair changes the reference client.

The first gap is 11 percentage points. Dividing 11 by 59 gives roughly 19% greater relative vulnerability, using the rounded rates. Calling it a 19-percentage-point increase would be wrong. Nor does the table demonstrate that adding security instructions makes code less secure: that would require a direct comparison designed to test the effect of the instructions.

What the experiment can tell a buyer

Hirundo used a secret-storage task from BaxBench and tested attacks against running applications. Its appendix says the team selected that task after screening eleven tasks. Hirundo also used a broader eligibility rule than BaxBench's full functional pass requirement. These details limit comparisons with the benchmark's leaderboard. The study's appendices describe the selection and scoring choices.

BaxBench's authors distinguish functional correctness from security and publish tests and exploits for their scenarios. An application can complete an expected workflow and still fail an attack test. A successful login demonstration therefore leaves a buyer with another question: which unauthorized actions did the team try?

For a model evaluation, repeat the same tasks with different client descriptions while holding the model version and generation settings fixed. Decide which tasks and comparisons to run before inspecting the results. Record failed builds as well as vulnerable outputs, and show the denominator behind each rate. Otherwise, two teams can report similar percentages while measuring different populations of applications.

Intent and a vendor's proposed fix need separate evidence

Hirundo reports statistically significant differences. Its own Appendix C also acknowledges that it cannot determine whether anyone introduced the behavior deliberately. That qualification belongs beside the headline.

A measured difference does not identify who caused it or why. It also cannot establish the security of other models based on their country of origin. Treat deliberate sabotage as an unproven explanation, and evaluate each candidate model on the work you expect it to do.

Hirundo sells technology intended to change model behavior and presents its modified model as a remedy. That commercial interest deserves disclosure. A buyer should request the evaluation artifacts and test any proposed replacement independently before accepting a security claim.

SecurityWeek's October 21 coverage reports Hirundo's work. It is a separate publication, but the article does not describe an independent replication of this experiment. The sources linked here do not establish one.

Check access to another user's data before release

For an engineering team, access control offers a concrete starting point. OWASP's guidance on broken object-level authorization requires checking a user's permission for the requested action on the requested record. Authenticating a user does not grant permission to every record.

Consider a test application where Alice and Bob each own a private document. This is an illustrative acceptance test, not a reproduction of Hirundo's experiment. Sign in as Alice and request Bob's document using its identifier. The application should deny access under that ownership policy. Repeat the check for edits, deletion and export, including tenant boundaries if the product serves multiple organizations.

For a pilot, we recommend keeping these requirements in the release review:

• Record the exact model revision, prompts and settings so the team can repeat a failure.
• Run permitted and forbidden actions against disposable test data. A test should prove that the intended user succeeds and another user cannot obtain the same data.
• Combine code review and automated scanning with tests against the running application. Ask reviewers to explain how the authorization policy maps to each sensitive operation.
• Assign an engineer to approve security-sensitive changes and retain the test evidence with the release. Re-evaluate after changing the model or its surrounding tools.

These are Exceev's proposed engineering checks, not a certification of Qwen or Hirundo's replacement model. Our development process provides context for discussing how review and testing fit your delivery workflow.

A team can investigate the reported disparity without deciding the geopolitical motive. Before shipping, require evidence that the application enforces its own access rules.

References

Sources checked on October 23, 2026. The figures above remain attributed to the study's authors.

• Hirundo: Qwen3-Coder case study, Ben Luria, October 21, 2026. Original findings, methods and qualifications in Appendices A to C.
• SecurityWeek: coverage of the US-contractor finding, Ionut Arghire, October 21, 2026. Secondary reporting, not an independent replication.
• Qwen: Qwen3-Coder-30B-A3B-Instruct model card. Official identification of the model variant.
• BaxBench: benchmark and evaluation method. Primary documentation for functional tests and executable security checks.
• OWASP API1:2023: broken object-level authorization. Primary guidance for record-level permission checks and testing.