Agweyu LLM (primary care, Kenya)

Generative AI-Enabled Clinical Decision Support System in Primary Care: A Pragmatic, Cluster-Randomized Trial

Patient / Population Intervention / Exposition Vergleich Endpunkt

In 9,691 patients at 16 primary care facilities in Kenya, whose 103 clinical officers were randomised to work with or without LLM assistance, a generative-AI (LLM) clinical decision support system embedded in the electronic medical record did not reduce treatment failure within 14 days compared with the electronic medical record alone.

N
9.691
Design
Pragmatic cluster RCT, clinical officers randomised, 16 primary care facilities, Kenya
Endpunkt
Expert-adjudicated composite of treatment-failure events within 14 days of enrolment
Relevanz
1
ErgebnisTreatment failure 102/4,693 (2.2%) vs 94/4,654 (2.0%) (aOR 0.77, 95% CI 0.55–1.08; P=0.13). No serious adverse events judged related to the intervention and no safety signal on independent review.
Agweyu A, et al. Nat Med. 2026;32(8):3032-3039. 10.1038/s41591-026-04503-6
Diskussion & Kritik

The first large pragmatic randomised trial of an LLM in real-world, low-resource primary care — and a null result on the patient-level primary outcome, with any benefit 'probably modest' in the authors' words. Reassuring safety, but a 2% event rate leaves little power, follow-up was 14 days, and the randomised units were clinicians rather than patients. A caution against extrapolating vignette-based LLM gains (Goh GPT-4 (management reasoning)) to patient outcomes.