Incident #157·Human-Computer Interaction

Amazon's Monitoring System Allegedly Pushed Delivery Drivers to Prioritize Speed over Safety, Leading to Crash

March 15, 20211 reportsView on AIID

Description

A lawsuit cited Amazon as liable in a crash involving its delivery driver, alleging that Amazon’s AI-powered driver monitoring system pushed drivers to prioritize speed over safety.

Actors

Alleged Deployers

amazon

Alleged Developers

amazon

Harmed Parties

amazon-workers
amazon-delivery-drivers

Classification

Risk Domain

Human-Computer Interaction

Risk Subdomain

5.1. Overreliance and unsafe use

MIT Risk Profile

AIUnintentionalPost-deployment

Source

Databaseaiid

LicenseCC BY-SA 4.0

View on AIID

Related Incidents (Human-Computer Interaction)

View all →

#826

Character.ai Chatbot Allegedly Influenced Teen User Toward Suicide Amid Claims of Missing Guardrails

Feb 2024

#67

Sleeping Driver on Tesla AutoPilot

Dec 2018

#17

Inappropriate Gmail Smart Reply Suggestions

Research Papers on This Topic

Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance

1,136 citationsJun 2020

Questioning the AI: Informing Design Practices for Explainable AI User Experiences

1,075 citationsJan 2020

Trust in AI and Its Role in the Acceptance of AI Technologies

957 citationsMar 2022

Metrics for Explainable AI: Challenges and Prospects

944 citationsDec 2018

Source Reports(1 total)

Amazon liable for crash because software “micromanages” delivery drivers, victim says

arstechnica.com· Nov 2021