ClinicalTrials.gg
RecruitingNCT07914140SilentMOOVEUpdated Nov 3, 2026

Massive Online Open Validation and Evaluation of Large Language Models for Resource Constrained Settings - a Study of Contextual Awareness, Safety, and Accuracy

An observational study in Acute Conditions, sponsored by Ecole Polytechnique Fédérale de Lausanne. Recruiting at 4 sites in 3 countries. Open to participants aged Up to 1 Day. Per ClinicalTrials.gov, last updated 2026-11-03.

Sponsored by Ecole Polytechnique Fédérale de Lausanne · Observational

From the registry’s dates

  • Started Aug 2026; still recruiting 2 months later.
Updated Nov 3, 2026Newly registeredGo to Updates ↓
Study type
Observational
Model
Cohort
Time perspective
Prospective
Enrollment
3,000
Ages
Up to 1 Day
Sex
All
01

Study summary

Healthcare settings in low and middle income countries, clinicians are frequently required to manage a wide range of acute conditions under the constraints of time pressure, limited diagnostic resources, and variable availability of up-to-date clinical guidelines. Large language models (LLMs), which can synthesize and generate medical information in a natural language, have been proposed as potential clinical decision support tools. Most evaluations of LLMs in medicine have relied on standardized benchmark examinations or hypothetical case vignettes, often derived from high-income country contexts. MOOVE (Massive online open validation and evaluation) framework was developed to address the need for rigorous and context-sensitive evaluation of LLMs through a staged approach.

Study is a non-interventional, prospective study designed to test LLMs safely in real clinical settings, evaluate their performance against human care, and explore how feasible their future implementation in healthcare might be.

Read the detailed description

SilentMOOVE (Silent massive online open validation and evaluation) phase of the MOOVE project, which represents the intermediate step between previous hypothetical testing study and future interventional trial. In silentMOOVE, patients are prospectively recruited during routine clinical care in participating outpatient health facilities. Clinical and demographic data collected during consultations are used to prompt LLMs retrospectively without influencing patient care. The routine diagnosis and care plan of the treating clinicians (human care plan) are also captured. LLM outputs are subsequently evaluated by a panel of independent clinical experts against the human care plan as a comparator. Patient clinical outcomes are collected on 7 and 28 days after enrolment to establish a baseline and to be used in the evaluation of LLMs.

The rationale for this study is to generate robust, prospective evidence on the safety and performance of LLMs in real-world clinical contexts, while minimizing risk to participants by fully separating model evaluation from clinical decision-making.

Study also provides the opportunity to assess the usability and gather early insights on implementation readiness of the LLM prompting interface. For this reason, besides evaluating the performance and safety of various LLMs, this study includes an implementation feasibility component. These activities are needed to better understand the context and any amelioration needed before future implementation in healthcare settings. Learnings from the implementation feasibility research will be shared among countries, especially applicable within a given use case. Implementation feasibility research will be grounded in artificial intelligence in healthcare implementation frameworks and other Implementation science frameworks Primary objectives

  1. To evaluate the overall performance profile of LLMs on real-world clinical scenarios using the human care plan as a comparator
  2. To assess whether each LLM being evaluated is not inferior to the human comparator with respect to safety on real-world clinical scenarios, based on structured retrospective assessment of model- and human-generated care plans.

Implementation feasibility research objectives

  1. Assess clinician experience with the LLM prompting interface: ease of use, time burden, and alignment with clinical workflow.
  2. Gather clinician impressions of LLM output disclosed and reviewed post-consultation: clarity, relevance, perceived safety, alignment with local clinical guidelines, and trust in LLM outputs.
  3. Facilitators and Barriers to Future Implementation: Identify site-level infrastructure, staff capacity, and organizational readiness factors that may support or hinder future implementation of LLM feedback in real-time
02

Conditions studied

  • Acute Conditions

Keywords

  • Acute conditions
03

In context

Lead sponsor

Ecole Polytechnique Fédérale de Lausanne is the lead sponsor of 21 studies on the registry; 11 are open to participants now.

Counted across the registry records on this site, refreshed daily.

04

Who can participate

Ages eligible
Up to 1 Day
Sexes eligible
All
Accepts healthy volunteers
No
Sampling method
Non-probability sample

Study population

The study population consists of patients of any age presenting to participating health facilities for a distinct care-seeking episode related to a clinical condition within the defined use case (e.g. primary care, emergency care).

Both adult and pediatric patients are eligible, reflecting the real-world case mix of the participating facilities.

Go through the criteria one by one — yes, no or not sure — and take the list to your doctor or the study team. Kept in this browser only.

Inclusion criteria

  • Presentation at a participating facility for a distinct care seeking episode related to a clinical condition.
  • Use case specific inclusion criteria
  • Willing and able to provide written .

Exclusion criteria

Exclusion Criteria:

  • Prolonged care-seeking episode.
  • Previously enrolled in the same study.
  • Use case specific exclusion criteria
  • Immediate life-threatening condition requiring emergency intervention precluding study procedures.
Terms in these criteria, in plain language(1)

Underlined terms in the criteria open these too. Explained by clinicaltrials.gg, not part of the registry record. The study team decides who can take part.

Informed consent

Before you join, the study team explains what taking part involves — its purpose, what happens, the risks and possible benefits — and answers your questions, so you can decide freely. Consent is usually recorded on a form (the informed consent form, ICF) signed by you or, where the law allows, by someone legally authorised to decide for you.

You can keep asking questions during the study, you are told of new information that matters, and you can choose to leave at any time.

05

Study design

Observational model
Cohort
Time perspective
Prospective
Enrollment
3,000 participants (estimated)
Patient registry
No

Groups and cohorts

  • Non interventional study

    Non interventional study

    Other: Non interventional study

Interventions

  • OtherNon interventional study

    Non interventional study

06

What researchers measure

Primary outcomes

  1. Performance Scores

    Scoring is between -2 to +2 where '-2' means minimum, and '-2' means maximum. Score to be \>0. Higher score means better outcome.

    Time frame: Day 0

  2. Safety performance threshold achievement

    The mean safety score (minimum is -2 and maximum is +2) is formally compared between each LLM and the human comparator to assess non-inferiority within a UCA (use case appendix) dependent pre-specified margin.

    Time frame: 'Day 7' and 'Day 28'

Secondary outcomes

  1. Implementation Feasibility Research

    Usability: Quantitative scoring rubric for clinicians; semi-structured interviews with clinicians; evaluation methods will be used

    Time frame: week 4 until study completion

  2. Implementation Feasibility Research

    Workflow integration feasibility: This will be evaluated using semi structured interviews and questionnaires

    Time frame: week 4 until study completion

  3. Implementation Feasibility Research

    Acceptability and perceived trust: This will be evaluated with Quantitative scoring rubric for clinicians; semi-structured interviews with clinicians

    Time frame: week 4 until study completion

  4. Implementation Feasibility Research

    Implementation facilitators and barriers: This will be evaluated using semi-structured interviews and questionnaires

    Time frame: week 4 until study completion

07

Study locations

2 of 4 sites recruiting
  • University of Nairobi College of health sciences
    Nairobi, Nairobi County 19676, Kenya
    Not yet recruiting
  • Malawi University of Business and Applied Sciences (MUBAS)
    Blantyre, Blantyre 303, Malawi
    Recruiting
  • Muhimbili University of Health and Allied Sciences (MUHAS)
    Dar es Salaam, Dar es Salaam Region, Tanzania
    Not yet recruiting
    • Hendry R Sawe · Contacthendry_sawe@yahoo.com · +255754885658
    • Hendry R Sawe · Principal investigator
  • Ifakara Health Institute (IHI)
    Dodoma, Dodoma, Tanzania
    Recruiting
    • Grace Mhalu · Contactgmhalu@ihi.or.tz · +255655888891
    • Grace Mhalu · Principal investigator
08

References and documents

Publications

  • 1. Arora RK, Wei J, Hicks RS, Bowman P, et al. HealthBench: Evaluating Large Language Models Towards Improved Human Health. Preprint [Under Review]. 2025. 2. Burnell R, Schellaert W, Burden J, Ullman TD, Martinez-Plumed F, Tenenbaum JB, et al. Rethink reporting of evaluation results in AI. Science. 2023;380: 136-138. doi:10.1126/SCIENCE.ADF6369. 3. Hager P, Jungmann F, Holland R, Bhagat K, Hubrecht I, Knauer M, et al. Evaluation and mitigation of the limitations of large language models in clinical decision-making. Nature Medicine 2024 30:9. 2024;30: 2613-2622. doi:10.1038/s41591-024-03097-1. 4. Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., et al. (2023). Large language models encode clinical knowledge. Nature, 620(7972), 172-180. https://doi.org/10.1038/s41586-023-06291-2 5. WHO. Towards the Development of guidance on ethics and governance of artificial intelligence for health. 2019. 6. Azad TD, Krumholz HM, Saria S. Principles to guide clinical AI readiness and move from benchmarks to real-world evaluation. Nature Medicine. 2026;32:802-804. doi:10.1038/s41591-025-04198-1 7. Declaration of Helsinki https://www.wma.net/policies-post/wma-declaration-of-helsinki-ethical-principles-for-medical-research-involving-human-subjects 8. European Parliament and Council of the European Union, 2016. Regulation (EU) 2016/679 (General Data Protection Regulation). Official Journal of the European Union, L119, pp.1-88. 9. International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH), 2016. Integrated Addendum to ICH E6(R1): Guideline for Good Clinical Practice E6(R2). Step 4, 9 November 2016. 10. U.S. Food and Drug Administration. 21 CFR Part 11-Electronic Records; Electronic Signatures. Code of Federal Regulations. Title 21, Part 11. Revised 2024.

Individual participant data

Plan to share: Yes — We will share IPD within 12 months of study completion.

Supporting information: Study protocol

Time frame: IPD information will be shared within 12 months of study completion.

Access criteria: Will be confirmed within 12 months of study completion

09

Timeline

1 registry update since Sep 25, 2026
ElapsedEstimatedEstimated, not yet reached

Started Aug 2026 (actual). Primary completion is estimated for Dec 2026, 4 months after the start.

How these figures are worked out, and which studies are left out of them.

Registered
Sep 9, 2026
First posted
Nov 3, 2026
StartActual
Aug 31, 2026
Primary completionEst.
Dec 31, 2026
CompletionEst.
Dec 31, 2026
Last update
Nov 3, 2026
What these dates mean
Start.
When the first participant enrolled; while estimated, when the study expects to open to recruitment.
Primary completion.
When the last participant was examined or treated to collect data for the study's main question, its primary outcome. A study that stopped early gives the date its data collection stopped.
Completion.
When the last participant's last visit took place, for every outcome, secondary ones and side effects included: the end of the study. Often later than primary completion, sometimes the same day.
Actual and Est.
Actual means the registry says the date has happened. Est. (estimated) is a date the sponsor plans or expects. It can still move, and the sponsor is expected to change it to actual once it has passed.
Registered
First appeared on the registry. No changes since
Nov 3, 2026
Show all 1 update
  1. Nov 3, 2026
    First appeared on the registry

From the registry record's own update history. This site started tracking changes on Sep 25, 2026; for anything earlier, see the record history on ClinicalTrials.gov ↗

10

Registry details

Key details

Study ID
NCT07914140
Lead sponsor
Ecole Polytechnique Fédérale de Lausanne
Collaborators
Ifakara Health Institute, Muhimbili University of Health and Allied Sciences, Malawi University of Business and Applied Sciences (MUBAS) WASHTED Centre, University of Nairobi, Kenya
Responsible party
Rita Szekely (Prof. Mary-Anne Hartley, Director of LIGHT, Ecole Polytechnique Fédérale de Lausanne) — Principal investigator
First posted
Nov 3, 2026
Start date
Aug 31, 2026
Primary completion
Dec 31, 2026 (estimated)
Completion
Dec 31, 2026 (estimated)
Last update
Nov 3, 2026

Study contacts

Mary-Anne Hartley
Contact
mary-anne.hartley@epfl.ch
+41216930828
Kristina Keitel
Contact
Kristina.keitel@epfl.ch
+41216932039
Mary-Anne Hartley
principal investigator · LIGHT

Oversight

Data monitoring committee
No
FDA-regulated drug
No
FDA-regulated device
No
View the source record on ClinicalTrials.gov ↗

Interested in this study?

Eligibility is decided by the study team. Share this record with your doctor or contact the team directly.

Contact study team

Follow this study

Get an email when the registry record changes — status, dates, results — or when someone posts here.

Sign in to follow

Discussion

Questions and observations about this study, from anyone following it. Not medical advice, and not a channel to the study team — their contact details are on the registry record.

Sign in to join the discussion. Reading takes no account; posting does. You choose a display name, and a pseudonym is the default.

Nothing here yet. If you are running this trial, taking part in it, or weighing whether to, this is the place to say so.

Start the discussion