AI2030 Responsible AI Tools Marketplace

Open Source Tools Commercial Tools Research Papers AI Incidents

Chicago Office Address
111 North Wabash Ave.
Chicago, IL 60602
United States

info@ai2030.org

Follow Us

AI2030

Home
About Us
Membership
Sponsorship

Useful Links

Get Tickets
Contact Us
Events

Quick Links

Blog
Speakers
Contacts

Marketplace

Open Source Tools
Commercial Tools
Research Papers
AI Incidents

© 2026 AI 2030 | All Rights Reserved

Privacy Policy|Terms of Use

Back to AI Incidents

Incident #475·AI Safety & Failures

McDonald's Reportedly Ends IBM Partnership After AI Drive-Thru Ordering Errors at U.S. Locations

June 2, 20216 reportsView on AIID

Explainability

Description

Between 2021 and mid-2024, McDonald's reportedly piloted an AI-enabled voice ordering system that was developed with IBM and based on its 2019 acquisition of Apprente. It was reportedly deployed at at over 100 U.S. drive-thrus. Social media posts alleged frequent misorders, such as adding unwanted items, mixing adjacent lane orders, and ignoring corrections. In June 2024, McDonald's confirmed it ended the IBM pilot, citing plans to explore alternative voice-AI vendors.

Actors

Alleged Deployers

mcdonald's

Alleged Developers

ibm

Harmed Parties

mcdonald's-customers

Classification

Risk Domain

AI Safety & Failures

Risk Subdomain

7.3. Lack of capability or robustness

MIT Risk Profile

AIUnintentionalPost-deployment

Topics

Explainability

Source

Related Incidents (AI Safety & Failures)

Chatbot Tessa gives unauthorized diet advice to users seeking help for eating disorders

May 2023

Australian Automated Debt Assessment System Issued False Notices to Thousands

Jul 2015

Amazon Alexa Responding to Environmental Inputs

Research Papers on This Topic

Concrete Problems in AI Safety

3,612 citationsJun 2016

A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI

2,338 citationsJul 2019

Representation Engineering: A Top-Down Approach to AI Transparency

1,499 citationsOct 2023

Explainable artificial intelligence (XAI) in deep learning-based medical image analysis

1,178 citationsJul 2021

Mitigation Tools

Tools related to this risk domain, via shared responsible AI topics.

Open Source

lime

marcotcr/lime

Lime: Explaining the predictions of any machine learning classifier

⭐ 12,100

facets

PAIR-code/facets

Visualizations for machine learning datasets

⭐ 7,374

interpret

interpretml/interpret

Fit interpretable models. Explain blackbox machine learning.

⭐ 6,788

captum

Source Reports(6 total)

260 McNuggets? McDonald’s Ends A.I. Drive-Through Tests Amid Errors

nytimes.com· Jun 2024

Bacon ice cream and nugget overload sees misfiring McDonald's AI withdrawn

bbc.com· Jun 2024

AI Incident Database (AIID), MIT AI Risk Initiative. Data from the AI Incident Database (AIID), MIT AI Risk Initiative. Licensed under CC BY-SA 4.0.

Databaseaiid

LicenseCC BY-SA 4.0

Dec 2015

pytorch/captum

Model interpretability and understanding for PyTorch

⭐ 5,548

Commercial

Datomize

Synthetic data generation for privacy compliance

Protect AI

Helping organizations build and use AI safely.

Censius

AI observability for model governance.

HiddenLayer

Model detection and response (MDR) for AI model protection.

McDonald's drive-thru robot made a mistake. Then it all got ridiculous

zdnet.com· Mar 2023

TikTokers are not lovin’ McDonald’s new AI-powered drive-thru

today.com· Feb 2023

TikTokers are roasting McDonald's hilarious drive-thru AI order fails — and it shows that robots won't take over restaurants any time soon

businessinsider.com· Feb 2023

'This is exactly why a person should take your order': Customer says McDonald's AI accidentally added 9 sweet teas to her order

dailydot.com· Feb 2023