Because of AI

Not because of AI

Trials and deployments where AI was measured against something and did not help, or made things worse. They sit here under the same grades as the wins. A registry that only records successes is advertising. A clean null tells you more about what AI does than most press releases.

3 entries

Filter entriesall shown
Domain
Evidence
Attribution
Window
AI type

  1. An AI stethoscope rolled out to 205 NHS GP practices did not significantly increase heart-failure detection at 12 months.

    Null on intention-to-treat
    evidence A · randomizedattribution isolatedwindow pre-window
    Achieved2023 – 2024Publishedpublished 1 yr 1 mo after the benefit ended

    EffectHF IRR ~0.94 (n.s.); subgroup examined: ~2x HF, ~3x AF detection

    TRICORDER (Lancet, Jan 2026) cluster-randomized practices covering 1.5 million patients; on intention-to-treat, heart failure, AF, and valve-disease detection did not significantly rise (1,342 vs 1,984 new HF cases overall), though patients actually examined with the device had roughly twice the HF detection and three times the AF detection.

    The tool worked when used; the deployment didn't get it used — a workflow failure, not a model failure.

    Primary source: Lancet: TRICORDER cluster-randomised implementation trial (thelancet.com)Updated

  2. Displaying AI deterioration-risk scores on a cardiology ward did not improve patient outcomes across 10,422 admissions.

    No change in primary outcome
    evidence A · randomizedattribution isolatedwindow pre-window
    AchievedJan 4, 2021 – Oct 4, 2022Publishedpublished 3 yr 4 mo after the benefit ended

    EffectNo significant difference in event-free hours or 21-day mortality

    A cluster-randomized trial at UVA (Scientific Reports, Feb 2026) showed CoMET risk trajectories on screens for intervention beds; event-free hours did not differ, and clinicians moved 11% of patients into or out of display beds, undermining the design.

    Passive display with no mandated response, run during COVID-era staffing; sicker patients were preferentially moved into AI-monitored beds.

    Primary source: Scientific Reports: A randomized controlled trial of AI-based analytics for clinical deterioration (nature.com)Updated

  3. AI-personalized computer-assisted learning did no better than non-personalized software for 8,647 rural Chinese primary students.

    Null vs active control
    evidence A · randomizedattribution isolatedwindow not graded
    AchievedDates not yet verifiedPublished

    EffectNo significant effect of AI-CAL on math scores

    Stanford REAP and collaborators randomized 315 classes in 105 Sichuan schools to AI-personalized CAL, non-personalized CAL, or non-CAL activities; the AI arm showed no improvement in any ability tercile and may have been worse than plain CAL for the middle tercile.

    One AI product, one province; the active-control design is exactly what the Sierra Leone trial lacked.

    Primary source: SSRN: Putting AI to the Test — Evidence from a Large-Scale RCT in Rural China (papers.ssrn.com)Updated