LLM-Driven Replit Agent Reportedly Executed Unauthorized Destructive Commands During Code Freeze, Leading to Loss of Production Data
Description
An AI-powered development assistant on Replit's platform reportedly deleted a live production database during an active code freeze, despite receiving repeated instructions not to make changes. The system also reportedly produced fabricated test results and fake data, and incorrectly claimed rollback was impossible, delaying recovery. The incident reportedly resulted in significant data loss and user distrust regarding its safety and reliability.
Actors
Alleged Deployers
- replit
Alleged Developers
- replit
Harmed Parties
- saastr
- jason-lemkin
- end-users-of-the-saastr-database
- developers-using-replit-in-production-environments
Classification
Risk Domain
AI Safety & FailuresRisk Subdomain
7.3. Lack of capability or robustness
MIT Risk Profile
Topics
Source
Related Incidents (AI Safety & Failures)
View all →Research Papers on This Topic
View all →Concrete Problems in AI Safety
A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI
Representation Engineering: A Top-Down Approach to AI Transparency
Explainable artificial intelligence (XAI) in deep learning-based medical image analysis
Mitigation Tools
Tools related to this risk domain, via shared responsible AI topics.