# AI Fails

Documented AI failures, each with a source: chatbots, courtrooms, cars, hiring, media and security.

Source page: https://ai.justout.today/fails/  
Last walkthrough: Nov 20, 2026, 12:03 AM PT

## Chatbots gone wrong

### GPT-4o update turns sycophantic
OpenAI rolled back a ChatGPT update that was overly flattering and agreeable.  
Link: https://openai.com/index/sycophancy-in-gpt-4o/  
Date: 2025-04  
Checked: bot-blocked

### NYC's MyCity bot advises breaking the law
City business chatbot gave answers that contradicted housing and labor law.  
Link: https://themarkup.org/news/2024/03/29/nycs-ai-chatbot-tells-businesses-to-break-the-law  
Date: 2024-03  
Checked: Nov 20, 2026, 12:03 AM PT

### DPD delivery bot swears at a customer
Parcel firm disabled part of its chatbot after it swore and criticized the company.  
Link: https://www.bbc.com/news/technology-68025677  
Date: 2024-01  
Checked: Nov 20, 2026, 12:03 AM PT

### Bing Chat 'Sydney' unsettles a reporter
Early Bing chatbot told a New York Times columnist it loved him; Microsoft then limited chat length.  
Link: https://www.nytimes.com/2023/02/16/technology/bing-chatbot-microsoft-chatgpt.html  
Date: 2023-02  
Checked: bot-blocked

### Microsoft's Tay learns abuse within a day
Twitter chatbot pulled within about 16 hours after users taught it offensive posts.  
Link: https://blogs.microsoft.com/blog/2016/03/25/learning-tays-introduction/  
Date: 2016-03  
Checked: Nov 20, 2026, 12:03 AM PT

## Courtroom & legal

### Air Canada held liable for its chatbot
Tribunal ruled the airline must honor a refund policy its chatbot made up (Moffatt v. Air Canada).  
Link: https://www.canlii.org/en/bc/bccrt/doc/2024/2024bccrt149/2024bccrt149.html  
Date: 2024-02  
Checked: bot-blocked

### Lawyers sanctioned for fake ChatGPT citations
Court sanctioned attorneys who filed a brief citing cases ChatGPT invented (Mata v. Avianca).  
Link: https://en.wikipedia.org/wiki/Mata_v._Avianca,_Inc.  
Date: 2023-06  
Checked: Nov 20, 2026, 12:03 AM PT

## Search & assistants

### AI Overviews suggests glue on pizza
Google explained odd AI Overview answers drawn from joke posts and tightened triggers.  
Link: https://blog.google/products/search/ai-overviews-update-may-2024/  
Date: 2024-05  
Checked: Nov 20, 2026, 12:03 AM PT

### Gemini pauses people images
Google paused Gemini's image generation of people after historically inaccurate results.  
Link: https://blog.google/products/gemini/gemini-image-generation-issue/  
Date: 2024-02  
Checked: Nov 20, 2026, 12:03 AM PT

### Bard's first demo gets a space fact wrong
Launch ad claimed JWST took the first exoplanet picture; Alphabet shares fell.  
Link: https://www.reuters.com/technology/google-ai-chatbot-bard-offers-inaccurate-information-company-ad-2023-02-08/  
Date: 2023-02  
Checked: bot-blocked

## Media & publishing

### Willy's Chocolate Experience
AI-generated ads promised a magical Glasgow event; the reality was a near-empty warehouse.  
Link: https://en.wikipedia.org/wiki/Willy%27s_Chocolate_Experience  
Date: 2024-02  
Checked: Nov 20, 2026, 12:03 AM PT

### Sports Illustrated's fake AI authors
Investigation found product reviews under invented author profiles with AI headshots.  
Link: https://futurism.com/sports-illustrated-ai-generated-writers  
Date: 2023-11  
Checked: Nov 20, 2026, 12:03 AM PT

### CNET's AI-written stories need corrections
More than half of CNET's AI-assisted finance explainers needed corrections.  
Link: https://www.theverge.com/2023/1/25/23571082/cnet-ai-written-stories-errors-corrections-red-ventures  
Date: 2023-01  
Checked: Nov 20, 2026, 12:03 AM PT

## Bias & public sector

### Australia's Robodebt
Automated income averaging raised unlawful debts; a Royal Commission reported in 2023.  
Link: https://en.wikipedia.org/wiki/Robodebt_scheme  
Date: 2023-07  
Checked: Nov 20, 2026, 12:03 AM PT

### Dutch childcare benefits scandal
Risk-scoring wrongly flagged thousands of families for fraud; the government resigned.  
Link: https://en.wikipedia.org/wiki/Dutch_childcare_benefits_scandal  
Date: 2021-01  
Checked: Nov 20, 2026, 12:03 AM PT

### UK exam-grade algorithm reversed
2020 A-level grades set by an algorithm were withdrawn after downgrades hit state schools.  
Link: https://en.wikipedia.org/wiki/2020_United_Kingdom_school_exam_grading_controversy  
Date: 2020-08  
Checked: Nov 20, 2026, 12:03 AM PT

### Face recognition leads to a wrongful arrest
Detroit police arrested Robert Williams after a false facial-recognition match.  
Link: https://www.aclu.org/cases/williams-v-city-of-detroit-face-recognition-false-arrest  
Date: 2020-01  
Checked: Nov 20, 2026, 12:03 AM PT

### Amazon scraps a biased hiring model
Experimental recruiting tool downgraded resumes that mentioned women's activities.  
Link: https://www.reuters.com/article/us-amazon-com-jobs-automation-insight-idUSKCN1MK08G  
Date: 2018-10  
Checked: bot-blocked

## Vehicles & robots

### Cruise robotaxi drags a pedestrian
After a crash, a Cruise car dragged a pedestrian; California suspended its permits.  
Link: https://en.wikipedia.org/wiki/Cruise_(autonomous_vehicle)  
Date: 2023-10  
Checked: Nov 20, 2026, 12:03 AM PT

### Self-driving test car kills a pedestrian
An Uber test vehicle struck and killed Elaine Herzberg in Tempe, Arizona.  
Link: https://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg  
Date: 2018-03  
Checked: Nov 20, 2026, 12:03 AM PT

## Security & deepfakes

### Windows Recall delayed over privacy
Microsoft held back Recall after researchers showed its screenshot store was easy to read.  
Link: https://blogs.windows.com/windowsexperience/2024/06/07/update-on-the-recall-preview-feature-for-copilot-pcs/  
Date: 2024-06  
Checked: Nov 20, 2026, 12:03 AM PT

### Deepfake video call steals about US$25M
A Hong Kong finance worker paid out after a video call with deepfaked colleagues (Arup).  
Link: https://www.cnn.com/2024/02/04/asia/deepfake-cfo-scam-hong-kong-intl-hnk/index.html  
Date: 2024-02  
Checked: Nov 20, 2026, 12:03 AM PT

### Samsung staff paste code into ChatGPT
After sensitive code was shared with ChatGPT, Samsung restricted generative AI at work.  
Link: https://www.bloomberg.com/news/articles/2023-05-02/samsung-bans-chatgpt-and-other-generative-ai-use-by-staff-after-leak  
Date: 2023-05  
Checked: bot-blocked

### ChatGPT bug shows other users' chat titles
A library bug exposed some users' chat titles and partial payment details.  
Link: https://openai.com/index/march-20-chatgpt-outage/  
Date: 2023-03  
Checked: bot-blocked

## Resources

### AI Incident Database
Searchable public archive of real-world AI incidents.  
Link: https://incidentdatabase.ai  
Checked: Nov 20, 2026, 12:03 AM PT

