Thursday, November 19, 2026 Updated Briefings Saved 中文
AI Daily Radar

AI and the frontier tech around it, explained every morning

Demonstrated

Bloomberg: AI models misidentified civilians as targets in simulated tests

Photo: Dietmar Rabich / Wikimedia Commons · CC BY-SA 4.0, cropped

Bloomberg’s Katrina Manson reported on November 17 that, in simulated scenarios, advanced AI models from Anthropic, Google, xAI, DeepSeek and OpenAI regularly misidentified civilians, ambulances and a school as military targets. The headline attributes the testing to an entity named “Vals”; the available excerpt does not identify the evaluators’ methodology, which model versions were tested, or how often each model erred. The account describes simulation results, not fielded systems, and does not say whether any military or defense user has deployed the models for targeting work.

Verified and unconfirmed

  • Verified: Bloomberg’s Katrina Manson reported on November 17 that, in simulated scenarios, advanced AI models regularly misidentified civilians, ambulances, surrendering soldiers and a school in Iran as military targets. According to Bloomberg and Bloomberg Law’s account of the independent research report behind the story, models from eight firms were tested, with Anthropic’s performing worst and OpenAI’s among the best but still making errors; the models named in the item include Anthropic, Google, xAI, DeepSeek and OpenAI.
  • Not yet confirmed: The evaluation’s design, the number of scenarios and runs, the scoring criteria, per-model error rates and the identity of the researchers are not established in the supplied material, which is brief re-reporting rather than the study itself. It is also unconfirmed whether these simulation results predict behavior in real targeting workflows, and no independent replication of the test has been reported.

Background

Vals, the San Francisco-based AI safety evaluator that ran the tests, published the evaluation on November 17, 2026, and counts parts of the US government among its clients; Bloomberg’s account and follow-up coverage say models from eight companies were tested, including the five named in the headline, and that Anthropic’s Claude performed worst. The scenarios were simulations rather than fielded weapons systems, so the results describe how the models respond to targeting-style prompts — including a school in Iran and surrendering soldiers — not how any deployed system behaves.

Why it matters

For anyone contemplating these models in imagery analysis or targeting workflows, the reported results argue against using model output as a target list without human review: the errors described are exactly the category — civilians, medical vehicles, protected sites — that the laws of armed conflict single out for protection. Because the report discloses no per-model error rates, teams cannot yet tell whether any one model is materially safer than another, so procurement or deployment decisions made on this account alone would rest on a single press summary.

Further reading

Comments

Comments

Comments use a GitHub account and are moderated. Comment rules

More in AI

See all