Planes were in the air and troops were ready to board a Chinese ship before the US learned an AI chatbot had misidentified its cargo, CNN reports

In the spring, in the early stages of the US-Iran war, the United States military came close to boarding a Chinese ship in the Middle East after an intelligence report claimed the vessel was carrying components of a nuclear weapons program. The report was false, and it had been produced with the help of an AI chatbot, CNN reported, citing four anonymous sources. One source said the incident “almost started a war.”

The military moved quickly to plan an intercept of the vessel, CNN reported. Two of the four sources said armed US personnel were preparing to board the ship, and the other two said military planes were already in the air. Shortly before the operation was set to begin, a special operations command analyst found that the report had been generated with AI.

A chatbot the original analyst used had inaccurately identified the material on the ship, according to CNN. CNN could not determine what the ship was actually carrying, and it is unclear whether the chatbot was a commercial product or a government-made one. One source called the report “entirely false.” Any kinetic action against a Chinese vessel would have carried a massive risk of spiraling into an armed conflict, the source noted.

In a statement, the Department of Defense said it remains committed to “rigorous verification of all intelligence inputs” and the responsible integration of new technologies.

The Pentagon’s push to adopt AI faster

The incident comes as the Pentagon pushes to become what it calls an “AI-first fighting force.” Under agreements announced in May with companies including Google, OpenAI, Amazon Web Services, Microsoft, SpaceX, Oracle, Nvidia and Reflection, the military can use their AI tools on classified networks for “lawful operational use,” the BBC reported.

The Pentagon has said access to a range of AI tools helps warfighters act with confidence, and that more than a million people across the department have used the platforms to cut processing times from months to days, per the BBC. The path to this integration has not been without controversy.

According to documents reviewed by The Intercept as part of a Freedom of Information lawsuit, the Department of Defense sought a version of OpenAI’s technology with “minimal refusal rates.” Heidy Khlaaf, chief scientist at the AI Now Institute, criticized the request as a step toward weakening safety guardrails.

OpenAI and the Pentagon have denied that such language was finalized in contracts. The Pentagon’s AI push also followed a public dispute with Anthropic, which had sought limits on the use of its AI for autonomous weapons and mass surveillance. The Pentagon later designated the company a supply chain risk, and the administration got into a showdown with Anthropic over it.

One former senior US official familiar with the military’s AI systems told CNN, “The internal tools are mostly just copies of the commercial stuff wearing lipstick.” CNN reported that analysts have long feared the technology could lead to a “catastrophic miscalculation” if it relies on poor or corrupted data.

One source familiar with current military policies said AI use in targeting is ramping up and that there is “no real guidance for how having a human in the loop will prevent civilian casualties or fratricide.” The administration has also set a goal to lead China in adoption of the technology.

In January, Defense Secretary Pete Hegseth released the agency’s “Artificial Intelligence Acceleration Strategy,” aiming to cut through bureaucracy and put advanced models into the hands of millions of personnel, per CNN. Hegseth stated, “We will unleash experimentation, eliminate bureaucratic barriers, focus our investments and demonstrate the execution approach needed to ensure we lead in military AI.”

The rollout is also decentralized, CNN reported, with different parts of the government using different tools under different orders and safety standards. One source familiar with the situation warned, “AI allows you to get to a bad idea faster.”