HomeSolutionsServicesResearchAccountabilityResourcesBook CompanyCommunityTEI

AI Incident Observatory

Curated analysis of publicly reported AI incidents to support institutional learning and risk prevention. The goal is not attribution or blame, but understanding how governance controls could have prevented failures.

Recent Incident Summaries

Illustrative examples from public reporting, anonymized and categorized by failure mode

Clinical AI Misclassification Critical

A clinical decision support system at a major U.S. hospital network missed a rare condition, delaying treatment due to edge case misclassification.

  • Domain: Healthcare
  • Failure mode: Edge case misclassification
  • Governance gap: No uncertainty escalation or documented human override
LLM Data Leakage High

An LLM chatbot at a Fortune 500 retailer disclosed confidential customer information through training data leakage in production responses.

  • Domain: E-commerce
  • Failure mode: Training data leakage
  • Governance gap: Insufficient output validation and access controls
Automated Trading Disruption Medium

An automated trading system on a cryptocurrency exchange triggered market disruption through autonomous action without human authority thresholds.

  • Domain: Finance
  • Failure mode: Autonomous action without constraints
  • Governance gap: No human authority thresholds or circuit breakers
Resume Screening Bias Medium

An HR technology provider's resume screening system exhibited systemic bias, amplifying historical patterns across protected categories.

  • Domain: Hiring
  • Failure mode: Bias amplification
  • Governance gap: Inadequate bias evaluation and limitation documentation

Incident Categories (2025 Review)

847 documented incidents categorized by failure mode across five primary areas

Bias and Fairness 296

Discriminatory outputs, disparate impact across protected groups, and amplification of historical bias in training data.

  • Hiring, lending, and insurance decisions
  • Content moderation disparities
  • Demographic performance gaps
Hallucinations and Overconfidence 237

Fabricated outputs presented with high confidence, including false citations, invented facts, and misleading recommendations.

  • Legal citation fabrication
  • Medical recommendation errors
  • Financial data misrepresentation
Security Exploits 152

Prompt injection, jailbreaking, adversarial inputs, and exploitation of model vulnerabilities in production systems.

  • Prompt injection and jailbreak attacks
  • Data extraction via adversarial queries
  • Model inversion and membership inference
Privacy Violations 102

Unauthorized disclosure of personal data, training data memorization, and failure to enforce data retention and consent boundaries.

  • Training data memorization leakage
  • Cross-context information exposure
  • Consent and retention violations
Safety and Physical Harm 60

Incidents where AI system failures resulted in or risked physical harm, including autonomous vehicle errors and medical device malfunctions.

  • Autonomous vehicle decision failures
  • Medical device misclassification
  • Industrial automation incidents
Prevention starts here AI Governance Toolkit Pre-deployment risk review, human authority design, uncertainty controls, and audit-ready documentation. Get the Free Toolkit →

Governance Insights

Cross-incident analysis based on comparative review of reported failures

Pre-Deployment Testing

Red-teaming could have mitigated approximately 89% of documented failures. Most incidents showed known failure patterns that were never tested before deployment.

  • Known failure patterns were not tested
  • Adversarial evaluation was absent
  • Edge cases were undocumented
Human in the Loop Controls

Systems with documented human authority showed approximately 73% fewer critical failures. Escalation pathways and override mechanisms correlated with fewer severe outcomes.

  • Documented escalation pathways
  • Human override mechanisms
  • Authority threshold enforcement
Output Validation

Independent output checks intercepted approximately 94% of hallucination type failures before user impact. Validation layers significantly reduce exposure to fabricated outputs.

  • Independent verification layers
  • Confidence calibration checks
  • Source attribution validation

Transparency Notice

This observatory does not provide real time surveillance, does not assign legal fault, and does not replace independent audits. It exists to support learning, prevention, and governance maturity. All incident summaries are anonymized and based on public sources. Figures reflect reported outcomes, not audited financial loss. These findings are analytical observations, not guarantees.