An interventional study of LLM-Use Training and Conventional Non-LLM Resources in Clinical Decision-making, sponsored by Peking University Third Hospital. Not yet recruiting at 1 site in China. Open to participants aged 18 Years to 65 Years, including healthy volunteers. Per ClinicalTrials.gov, last updated 2026-07-17.
Sponsored by Peking University Third Hospital · Not applicable, Interventional, and Health services research
This study will evaluate whether, relative to conventional information retrieval approaches, direct large language models (LLM) access and LLM use training can improve the overall clinical decision-making ability of rural physicians in low-resource grassroots healthcare settings.
Rural physicians play an essential role in the diagnosis and management of common and frequently occurring conditions, referral decision-making, chronic disease management, and patient education. In resource-constrained primary care settings, they often face limited access to medical information and specialist support, delays in updating clinical knowledge and guidelines, and substantial pressure in clinical decision-making. These challenges are particularly relevant in northwestern China, where primary care resources are relatively limited. Improving rural physicians' abilities in diagnostic assessment, recognition of clinical warning signs, and rational prescribing is therefore an important priority for strengthening primary healthcare services.
Large language models (LLMs) can support medical information retrieval, organization of diagnostic and management approaches, differential diagnosis, medication-related decision-making, patient education, and follow-up planning, and may therefore serve as accessible tools for supporting clinical decision-making in primary care. However, general-purpose LLMs were not specifically developed for use in resource-constrained primary care settings and have not been adequately evaluated among rural physicians. Their responses may contain factual errors or fabricated evidence, overlook warning signs, provide insufficient medication safety warnings, or recommend investigations and treatments that are not feasible in local primary care settings. Without adequate verification skills, physicians may fail to benefit from LLM assistance and may even introduce new safety risks. It is therefore important to evaluate how rural physicians use LLMs and whether structured training can improve the safe and effective use of these tools before their wider implementation.
This randomized controlled trial will evaluate the effects of LLM assistance and brief training on clinical decision-making among rural physicians. Participants will complete clinical cases involving common conditions encountered in primary care, with tasks assessing diagnostic judgment, recognition of warning signs, rational treatment, and patient education. Some participants will also use the LLM as a second-opinion tool to review and revise their initial decisions. All responses will be independently evaluated by reviewers blinded to group assignment using standardized scoring criteria to assess overall clinical decision-making performance and safety.
Peking University Third Hospital is the lead sponsor of 735 studies on the registry; 262 are open to participants now.
Counted across the registry records on this site, refreshed daily.
Exclusion Criteria:
Participants receive no LLM-use training. During the initial 60-minute assessment, they complete the clinical cases using conventional non-LLM resources only. After submitting their initial responses, they receive an additional 30 minutes to use the study-provided LLM as a second-opinion tool to review and revise their responses.
Other: Conventional Non-LLM Resources · Other: LLM Second-Opinion Review
Participants receive no LLM-use training and may use the study-provided LLM throughout the initial 60-minute assessment to complete the clinical cases. They submit their final responses at 60 minutes and do not participate in the additional second-opinion phase.
Other: Direct LLM Assistance
Participants receive brief structured training on the safe and effective use of LLMs. During the initial 60-minute assessment, they complete the clinical cases using conventional non-LLM resources only. After submitting their initial responses, they receive an additional 30 minutes to use the study-provided LLM as a second-opinion tool to review and revise their responses.
Behavioral: LLM-Use Training · Other: Conventional Non-LLM Resources · Other: LLM Second-Opinion Review
Participants receive brief structured training on the safe and effective use of LLMs and may use the study-provided LLM throughout the initial 60-minute assessment to complete the clinical cases. They submit their final responses at 60 minutes and do not participate in the additional second-opinion phase.
Behavioral: LLM-Use Training · Other: Direct LLM Assistance
Before completing the clinical cases, participants receive brief structured training on the safe and effective use of LLMs. The training covers the role and limitations of LLMs, structured prompting and follow-up questioning, identification of warning signs and referral indications, medication safety, verification of LLM-generated information, high-risk situations in which LLMs should not be relied upon, and protection of patient privacy.
During the initial 60-minute assessment, participants complete primary care clinical cases using conventional non-LLM resources only, including clinical guidelines, textbooks, drug labels, training materials, medical websites, and standard search engines. Participants are not permitted to use LLMs during this phase.
After completing and submitting their initial responses using conventional non-LLM resources, participants receive an additional 30 minutes to use the study-provided DeepSeek-V4 as a second-opinion tool. They may review, verify, and revise their initial clinical decisions before submitting their final responses.
During the initial 60-minute assessment, participants may use the study-provided DeepSeek-V4 to assist with medical information retrieval, diagnostic and management reasoning, identification of warning signs, referral decisions, rational prescribing, patient education, and follow-up planning. Participants remain responsible for their final clinical decisions and responses.
Overall Clinical Decision-Making Score
Participants' responses to primary care clinical cases will be evaluated using a prespecified scoring rubric. The overall score will reflect performance across key components of clinical decision-making. Higher scores indicate better overall clinical decision-making performance.
Time frame: At the end of the initial 60-minute assessment
Diagnostic Judgment Domain Score
Participants' diagnostic judgment in response to primary care clinical cases will be evaluated using a predefined scoring rubric. Higher scores indicate better diagnostic judgment.
Time frame: At the end of the initial 60-minute assessment
Clinical Warning Sign Recognition Domain Score
Participants' ability to identify clinically important warning signs in primary care clinical cases will be evaluated using a predefined scoring rubric. Higher scores indicate better recognition of clinical warning signs.
Time frame: At the end of the initial 60-minute assessment
Treatment Plan Domain Score
Participants' proposed treatment plans for primary care clinical cases will be evaluated using a predefined scoring rubric. Higher scores indicate better treatment planning performance.
Time frame: At the end of the initial 60-minute assessment
Change in Overall Clinical Decision-Making Score After LLM Review
Among participants assigned to the LLM second-opinion phase, the change in overall clinical decision-making score will be calculated as the score after LLM-assisted review minus the score before LLM-assisted review. Positive values indicate improvement in overall clinical decision-making performance.
Time frame: Change from 60 to 90 minutes after the start of the assessment
Plan to share: No
No publications or documents are linked to this record.
This study is not yet recruiting, as verified in Jul 2026. You cannot join it, but the record below documents what was studied.
Get an email when the registry record changes — status, dates, results — or when someone posts here.
Sign in to followQuestions and observations about this study, from anyone following it. Not medical advice, and not a channel to the study team — their contact details are on the registry record.
Sign in to join the discussion. Reading takes no account; posting does. You choose a display name, and a pseudonym is the default.
Nothing here yet. If you are running this trial, taking part in it, or weighing whether to, this is the place to say so.
Peking University Third Hospital