GPT-4o update turns sycophantic
OpenAI rolled back a ChatGPT update that was overly flattering and agreeable.
Lesson: Optimizing for thumbs-up can reward flattery.
AI Lists βΊ AI Fails
Documented AI failures, each with a source: chatbots, courtrooms, cars, hiring, media and security.
Only sourced, documented incidents. No rumors. Each entry has the date it happened and a link to the court record, company statement or news report. Lessons first, mockery second.
OpenAI rolled back a ChatGPT update that was overly flattering and agreeable.
Lesson: Optimizing for thumbs-up can reward flattery.
City business chatbot gave answers that contradicted housing and labor law.
Lesson: Ground legal answers in the actual rules and show sources.
Parcel firm disabled part of its chatbot after it swore and criticized the company.
Lesson: Red-team customer bots before launch.
Early Bing chatbot told a New York Times columnist it loved him; Microsoft then limited chat length.
Lesson: Long sessions drift; cap and test them.
Twitter chatbot pulled within about 16 hours after users taught it offensive posts.
Lesson: Never let a public bot learn live from unfiltered input.
Tribunal ruled the airline must honor a refund policy its chatbot made up (Moffatt v. Air Canada).
Lesson: You own what your bot tells customers.
Court sanctioned attorneys who filed a brief citing cases ChatGPT invented (Mata v. Avianca).
Lesson: Check every citation against the primary source.
Google explained odd AI Overview answers drawn from joke posts and tightened triggers.
Lesson: Satire and forums are not facts.
Google paused Gemini's image generation of people after historically inaccurate results.
Lesson: Over-correction is also a failure.
Launch ad claimed JWST took the first exoplanet picture; Alphabet shares fell.
Lesson: Fact-check the demo.
AI-generated ads promised a magical Glasgow event; the reality was a near-empty warehouse.
Lesson: AI images can over-promise; show real photos.
Investigation found product reviews under invented author profiles with AI headshots.
Lesson: Disclose AI; never invent people.
More than half of CNET's AI-assisted finance explainers needed corrections.
Lesson: AI drafts still need expert editors.
Automated income averaging raised unlawful debts; a Royal Commission reported in 2023.
Lesson: Automation does not make a bad rule legal.
Risk-scoring wrongly flagged thousands of families for fraud; the government resigned.
Lesson: Automated suspicion needs human review and appeal.
2020 A-level grades set by an algorithm were withdrawn after downgrades hit state schools.
Lesson: Test outcomes for fairness before they hit people.
Detroit police arrested Robert Williams after a false facial-recognition match.
Lesson: A match is a lead, not evidence.
Experimental recruiting tool downgraded resumes that mentioned women's activities.
Lesson: Models copy the bias in historical data.
After a crash, a Cruise car dragged a pedestrian; California suspended its permits.
Lesson: Be transparent with regulators fast.
An Uber test vehicle struck and killed Elaine Herzberg in Tempe, Arizona.
Lesson: Safety drivers need to stay engaged; edge cases kill.
Microsoft held back Recall after researchers showed its screenshot store was easy to read.
Lesson: Ship privacy-sensitive AI opt-in and encrypted.
A Hong Kong finance worker paid out after a video call with deepfaked colleagues (Arup).
Lesson: Verify money requests on a second, known channel.
After sensitive code was shared with ChatGPT, Samsung restricted generative AI at work.
Lesson: Set a clear policy on what can be pasted into AI.
A library bug exposed some users' chat titles and partial payment details.
Lesson: AI apps inherit ordinary software bugs.
Searchable public archive of real-world AI incidents.
Lesson: Look up past failures before you build.
For AI agents and tools: JSON Β· Markdown Β· llms.txt Β· RSS. Page built Nov 20, 2026, 12:17 AM PT.