An interventional study of GPT-4 in Clinical Decision-making, sponsored by Stanford University. Completed at 1 site in United States. Per ClinicalTrials.gov, last updated 2024-09-27.
Sponsored by Stanford University · Not applicable, Interventional, and Treatment
This study will evaluate the effect of providing access to GPT-4, a large language model, compared to traditional management decision support tools on performance on case-based management reasoning tasks.
Artificial intelligence (AI) technologies, specifically advanced large language models like OpenAI's ChatGPT, have the potential to improve medical decision-making. Although ChatGPT-4 was not developed for its use in medical-specific applications, it has demonstrated promise in various healthcare contexts, including medical note-writing, addressing patient inquiries, and facilitating medical consultation. However, little is known about how ChatGPT augments the clinical reasoning abilities of clinicians.
Clinical reasoning is a complex process involving pattern recognition, knowledge application, and probabilistic reasoning. Integrating AI tools like ChatGPT-4 into physician workflows could potentially help reduce clinician workload and decrease the likelihood of mismanagement. However, ChatGPT-4 was not developed for clinical reasoning nor has it been validated for this purpose. Further, it may be subject to disinformation, including convincing confabulations that may mislead clinicians. If clinicians misuse this tool, it may not improve reasoning and could even cause harm. Therefore, it is important to study how clinicians use large language models to augment clinical reasoning prior to routine incorporation into patient care.
In this study, participants will be randomized to answer clinical management cases with or without access to ChatGPT-4. Each case has multiple components, and the participants will be asked to discuss their reasoning for each component. Answers will be graded by independent reviewers blinded to treatment assignment. A grading rubric was developed for each case by a panel of 4-7 expert discussants. Discussants independently developed a rubric for each case, and then any discrepancies were resolved through multiple rounds of discussions.
Stanford University is the lead sponsor of 2,117 studies on the registry; 425 are open to participants now.
Of its 259 completed or terminated interventional studies of FDA-regulated products, 197 (76%) have results posted.
Counted across the registry records on this site, refreshed daily.
Exclusion Criteria:
Group will be given access to GPT-4
Other: GPT-4
Group will not be given access to GPT-4 but will be encouraged to use any resources they wish besides large language models (UpToDate, Dynamed, google, etc).
OpenAI's GPT-4 large language model with chat interface.
Management Reasoning
Percent correct (range: 0 to 100) for each case.
Time frame: Within one-hour study
Time Spent on Management
Time (in minutes) participants spend per case between the two study arms.
Time frame: Within one-hour study
Plan to share: No
No publications or documents are linked to this record.
This study is completed, as verified in Sep 2024. You cannot join it, but the record below documents what was studied.
Get an email when the registry record changes — status, dates, results — or when someone posts here.
Sign in to followQuestions and observations about this study, from anyone following it. Not medical advice, and not a channel to the study team — their contact details are on the registry record.
Sign in to join the discussion. Reading takes no account; posting does. You choose a display name, and a pseudonym is the default.
Nothing here yet. If you are running this trial, taking part in it, or weighing whether to, this is the place to say so.
Stanford University