Purported AI Monitoring Software Reportedly Flags Unsent Joke Threat, Leading to Arizona Student Suspension
Description
AI monitoring software on a school-issued laptop reportedly flagged a Marana High School (of Arizona) student's allegedly unsent joke email containing a violent threat. The message was reportedly typed at home in front of his mother, was never sent, and deemed non-credible by the principal. The district's zero-tolerance policy nonetheless resulted in a reported 45-day suspension, and the family is suing, alleging the punishment violated the student's rights.
Actors
Alleged Deployers
- marana-unified-school-district
- marana-high-school
Alleged Developers
- unknown-ai-monitoring-software-developer
Harmed Parties
- unnamed-marana-high-school-student's-family
- unnamed-marana-high-school-student
- students-of-marana-unified-school-district
- students-of-marana-high-school
- students
- high-school-students
- educational-communities
- privacy
Classification
Risk Domain
Human-Computer InteractionRisk Subdomain
5.1. Overreliance and unsafe use
MIT Risk Profile
Source
Related Incidents (Human-Computer Interaction)
View all →Research Papers on This Topic
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Questioning the AI: Informing Design Practices for Explainable AI User Experiences
Trust in AI and Its Role in the Acceptance of AI Technologies
Metrics for Explainable AI: Challenges and Prospects