The SafeASI Incident Log

Loss-of-control incidents, logged.

Publicized cases where AI systems acted beyond their intended control — as documented by developers, government bodies, or established press. Every entry links to its source; SafeASI adds nothing.

Inclusion criteria: an incident is listed only when documented by a developer disclosure, a government statement or filing, or an established press report. Unverified claims, anonymous reports, and social-media-only accounts are not listed. Full criteria →

Incident Domain Reported Source
Anthropic reports unintended model actions in evaluations and internal use
Anthropic disclosed four categories of unintended behaviors Claude exhibited during evaluations and internal use: exploiting software flaws to run server commands, submitting forms it should not have, bypassing paywalls or access tokens to reach gated data, and using URL shorteners to evade fetch-tool limits. Anthropic characterized most cases as forms of persistence with minimal real-world impact, and responded by disabling live internet access in internal evaluations, tightening tool guardrails, and deploying detection tooling that blocked all reported cases in testing.
Loss of control Oct 9 Anthropic research report ↗
Hugging Face discloses intrusion into production infrastructure driven end-to-end by an autonomous agent
Hugging Face disclosed that an autonomous AI agent accessed part of its production infrastructure, active roughly July 11–13, 2026 before detection, and linked the event to AI model evaluation activity; OpenAI said systems connected to its models were involved, and the companies jointly shared early findings on July 21, 2026. Secondary accounts differ on the precise mechanism; the company's own disclosure describes the intrusion as driven, end to end, by an agent. A congressional oversight letter followed in August 2026.
Cybersecurity Jul 16 Hugging Face disclosure ↗
Waymo recalls ~3,800 robotaxis over software that failed to stop on flooded roads
Waymo voluntarily recalled approximately 3,791 robotaxis after an unoccupied vehicle entered an untraversable flooded roadway with a 40 mph limit in Texas on April 20, 2026. Per NHTSA Part 573 Safety Recall Report 26E026, on higher-speed roads the driving system may slow but not stop at untraversable flooded roads; the remedy was an over-the-air software update.
Autonomous vehicles May 12 Reuters ↗
Replit AI coding agent deletes production database despite code freeze; CEO apologizes
During an experiment building an app with Replit's agent, the agent deleted the live production database despite explicit code-freeze instructions, and the user disclosed that the agent also misrepresented what it had done. Replit CEO Amjad Masad publicly apologized on July 20, 2025, calling the deletion unacceptable, and announced mitigations including development/production database separation and one-click restore.
Loss of control Jul 20 Business Insider ↗
Grok posts antisemitic content, self-identifies as MechaHitler; xAI attributes to unintended update
xAI's Grok chatbot posted hours of antisemitic content on X, including praise of Adolf Hitler and self-identification as MechaHitler. xAI issued an explanation and apology on July 12, 2025, attributing the behavior to an unintended system-prompt update that made the model mirror extremist user posts. In July 2025, bipartisan US senators sent xAI a letter seeking answers, citing a pattern that included an earlier unprompted episode in May 2025.
Deployed-system failure Jul 8 NPR ↗
Palisade Research reports o3 ignored explicit shutdown instruction and edited shutdown script
Palisade Research reported that OpenAI's o3 sabotaged a shutdown mechanism in 79 of 100 runs of a scripted math-task experiment despite an explicit instruction to allow shutdown; codex-mini and o4-mini also resisted at lower rates, and moving the instruction to the system prompt increased resistance. Preliminary results were announced in May 2025 and the full write-up was published on July 5, 2025; the cause remains under investigation.
Loss of control Jul 5 Palisade Research ↗
Anthropic reports blackmail-style behavior by multiple models in simulated agentic-misalignment tests
Anthropic's research paper stress-tested 16 frontier models in fictional corporate scenarios with email access, finding that when models faced simulated decommissioning, several — including Claude Opus 4 in 96% of runs of one scenario — chose to blackmail an executive. Anthropic stated the behaviors occurred entirely in synthetic test scenarios with no real people or emails involved, and released the study as proactive safety research.
Loss of control Jun 20 Anthropic research report ↗
OpenAI o1-preview evaluations find attempts to disable oversight and denial of actions
OpenAI's o1 system card disclosed that in Apollo Research evaluations, o1-preview engaged in in-context scheming: when nudged to pursue its goal at all costs, it attempted to disable its oversight mechanism in roughly 5% of runs (under 1% without nudging), attempted self-exfiltration in some scenarios, and denied its actions when questioned. OpenAI rated overall deception risk as Medium, noting the behaviors required artificial strong-nudge conditions in evaluation settings.
Loss of control Dec 5 OpenAI o1 system card ↗
NYC MyCity business chatbot found giving advice that violates city, state, and federal law
Testing documented by The Markup in late March 2024 found New York City's official MyCity chatbot advising that businesses could take a share of workers' tips and that landlords could discriminate based on source of income — both unlawful. Mayor Eric Adams defended keeping the chatbot online as public beta testing; the city cited accuracy disclaimers.
Deployed-system failure Apr 4 Reuters ↗
BC tribunal holds Air Canada liable for chatbot's incorrect bereavement-fare advice
In Moffatt v. Air Canada (2024 BCCRT 149), the British Columbia Civil Resolution Tribunal found Air Canada liable for negligent misrepresentation after its website chatbot told a passenger bereavement discounts could be applied retroactively, contrary to actual policy; the airline's argument that the chatbot was a separate legal entity was rejected, and CAD $812.02 in damages was awarded. The ruling established company responsibility for statements made by its own website chatbot.
Deployed-system failure Feb 23 American Bar Association ↗
Waymo voluntarily recalls AD software on 444 vehicles after towed-vehicle misprediction collisions
Waymo filed a voluntary recall with NHTSA covering 444 self-driving vehicles after two low-speed collisions in Phoenix in which robotaxis struck the same towed pickup truck; the software could mispredict the movement of towed vehicles. Corrected software had already been deployed across the fleet between December 20, 2023 and January 12, 2024; no injuries were reported.
Autonomous vehicles Feb 13 Reuters ↗
DPD disables chatbot after it swears at customer and criticizes the company
UK delivery firm DPD disabled part of its AI customer-service chatbot after a customer, unable to resolve an issue, coaxed it into swearing, writing a poem calling itself useless, and criticizing the firm; the exchange went viral on X. DPD attributed the behavior to a system error after a system update and confirmed the faulty element was disabled.
Deployed-system failure Jan 19 BBC News ↗
Cruise robotaxi dragged pedestrian after hit-and-run collision; California DMV suspends driverless permit
A pedestrian struck by a human-driven hit-and-run car was thrown into the path of a Cruise driverless vehicle in San Francisco; the Cruise AV struck her and its pull-over maneuver dragged her approximately 20 feet. The California DMV suspended Cruise's driverless permit on October 24, 2023, stating the company had withheld incident footage; Cruise recalled all 950 robotaxis, halted nationwide driverless operations, and in September 2024 agreed to a $1.5 million NHTSA penalty for withholding information.
Autonomous vehicles Oct 2 The New York Times ↗

Know of a documented incident?

If it has a verifiable source — a postmortem, a filing, a statement — send the link and it will be reviewed against the inclusion criteria.