ClinicalTrials.gg
CompletedNCT06229379Updated Nov 22, 2024

The Effects of a Large Language Model on Clinical Questioning Skills

An interventional study of "Digital twin patient" and Interaction with real patients in Cataract, Glaucoma and Diabetic Retinopathy, sponsored by Sun Yat-sen University. Completed at 1 site in China. Open to participants aged 18 Years to 25 Years, including healthy volunteers. Per ClinicalTrials.gov, last updated 2024-11-22.

Sponsored by Sun Yat-sen University · Not applicable, Interventional, and Other

Phase
Not applicable
Study type
Interventional
Enrollment
84
Allocation
Randomized
Ages
18 Years to 25 Years
Sex
All
01

Where this study stands

Counted across the records on this site · 2026-11-12

Among 52 studies in Cataract by Sun Yat-sen University — 3 open to participants now, 4 under way, none of them phase 3. Cataract · Sun Yat-sen University.

Enrollment
18th of 35
84 · median 84
Started
2023
with 3 others
No phase
35 of 52
2 open now
Registered
52nd of 52
by date

Among the 52 studies in Cataract by Sun Yat-sen University, this one is the 18th largest by enrollment of the 35 interventional ones, is one of 4 that started in 2023, is one of 35 with no phase, and was the 52nd registered.

By phase

52 studies; the count open now beside each. This study's row is marked.

No phasethis study35 · 2 open
Observational17 · 1 open

By the year they started

First in 2005; this study’s year is marked. 4 started in 2023.

What they test

Size and results

Median enrollment 84 across the 35 interventional studies, 428 across the 17 observational. Of the 32 that completed or were terminated, 2 (6%) have results posted.

Was this useful?
Watch this landscapeDownload CSVWith an account: a weekly mail of what changed among 52 studies in Cataract by Sun Yat-sen University, or all of them as a file.
02

Study summary

The researchers have used the ophthalmology textbook, clinical guideline consensus, the Internet conversation data and knowledge base of Zhongshan Ophthalmology Center in the early stage, combined with artificial feedback reinforcement learning and other techniques to fine-tune and train the LLM, and developed "Digital Twin Patient", a localized large language model that has the ability to answer ophthalmology-related medical questions, and also constructed a combination of automated model evaluation and manual evaluation by medical experts. The evaluation system combining automated model evaluation and manual evaluation by medical experts was constructed at the same time.

This project intends to integrate "Digital Twin Patient" into undergraduate ophthalmology apprenticeship, simulate the consultation process of real patients through the online interaction between students and "Digital Twin Patient", explore the effect of "Digital Twin Patient" consultation teaching, provide emerging technology tools for guiding medical students to actively learn a variety of ophthalmology cases, cultivate clinical thinking, and provide the possibility of creating a new mode of intelligent teaching.

Read the detailed description

At present, the main form of clinical questioning skills teaching is to let undergraduates who participate in the apprenticeship first learn the characteristics and diagnostic points of cases, and then practice questioning on real patients in the wards. However, due to the large number of trainee students, it is difficult to meet the teaching demand in terms of the number of cases available for questioning and the richness of disease types under the current teaching mode. Therefore, it is necessary to utilize new intelligent technologies and create a new model of questioning skills teaching to improve teaching efficiency and enhance students' clinical thinking.

Large-scale language modeling (LLM) is a deep learning technology that can learn knowledge from a large amount of text, and AI chatbots such as ChatGPT are a typical example of its application. AI chatbots are characterized by anthropomorphic comprehension and diversified natural language generation abilities in different contexts, and have been initially applied in the medical field, such as passing the U.S. Medical Licensing Examination, assisting in ophthalmic history documentation and answering ophthalmic questions. However, it has been found that although LLM has fair modeling performance in general medical knowledge, it still needs to be improved in the area of specialty diseases. Based on this, the researcher's team has used the ophthalmology textbook, clinical guideline consensus, the Internet conversation data and knowledge base of Zhongshan Ophthalmology Center in the early stage, combined with artificial feedback reinforcement learning and other techniques to fine-tune and train the LLM, and developed "Digital Twin Patient", a localized large language model that has the ability to answer ophthalmology-related medical questions, and also constructed a combination of automated model evaluation and manual evaluation by medical experts. The evaluation system combining automated model evaluation and manual evaluation by medical experts was constructed at the same time.

This project intends to integrate "Digital Twin Patient" into undergraduate ophthalmology apprenticeship, simulate the consultation process of real patients through the online interaction between students and "Digital Twin Patient", explore the effect of "Digital Twin Patient" consultation teaching, provide emerging technology tools for guiding medical students to actively learn a variety of ophthalmology cases, cultivate clinical thinking, and provide the possibility of creating a new mode of intelligent teaching.

03

Conditions studied

  • Cataract
  • Glaucoma
  • Diabetic Retinopathy
  • Keratitis
  • Conjunctivitis

Keywords

  • Large language model
  • Clinical education
  • Ophthalmology
04

Who can participate

Ages eligible
18 Years to 25 Years
Sexes eligible
All
Accepts healthy volunteers
Yes

Go through the criteria one by one — yes, no or not sure — and take the list to your doctor or the study team. Kept in this browser only.

Inclusion criteria

  • All undergraduate students from Sun Yat-sen University who participate in the ophthalmological internship.

Exclusion criteria

Exclusion Criteria:

  • Students who refuse to sign .
Terms in these criteria, in plain language(1)

Underlined terms in the criteria open these too. Explained by clinicaltrials.gg, not part of the registry record. The study team decides who can take part.

Informed consent

Before you join, the study team explains what taking part involves — its purpose, what happens, the risks and possible benefits — and answers your questions, so you can decide freely. Consent is usually recorded on a form (the informed consent form, ICF) signed by you or, where the law allows, by someone legally authorised to decide for you.

You can keep asking questions during the study, you are told of new information that matters, and you can choose to leave at any time.

05

Study design

Phase
Not applicable
Primary purpose
Other
Allocation
Randomized
Intervention model
Parallel assignment
Masking
None (open label)
Enrollment
84 participants (actual)

Study arms

  • Experimental
    "Digital twin patient"

    The students in the "Digital twin patient" group were trained in history taking using a "digital twin patient" on the first day of a training program, and then took a 15-minute clinical questioning exam using the "digital twin patient" on the second day.

    Device: "Digital twin patient"

  • Other
    Real patient

    The students in the Real patient group were trained in history taking using a real patient on the first day of a training program, and then took a 15-minute clinical questioning exam using the "digital twin patient" on the second day.

    Device: "Digital twin patient" · Behavioral: Interaction with real patients

Interventions

  • Device"Digital twin patient"

    "Digital twin patient" can serve as patients with specific diseases for medical students to acquire disease history and thus practice clinical questioning skills.

  • BehavioralInteraction with real patients

    As in traditional medical education, medical students need to interact with real patients to practice history collection skills.

06

What researchers measure

Primary outcomes

  1. Students' scores in the medical history acquisition exam

    Time frame: Weekly during this study (up to 10 months)

07

Study locations

1 site
  • Zhongshan Ophthalmic Center, Sun Yat-sen Univerisity
    Guangzhou, Guangdong 510060, China
08

References and documents

Individual participant data

Plan to share: No

No publications or documents are linked to this record.

09

Timeline

Tracking since Sep 25, 2026
ElapsedEstimatedEstimated, not yet reachedSimilar studies: half by the dot; the band is the middle halfHalf of similar studies reached it by the dot; the band is the middle half, from a quarter to three quarters

Ran from Nov 2023 to primary completion in May 2024, 5 months.

Of 300 non-industry interventional studies with no phase under Cataract, half reached primary completion within 18 months of starting (the median), counting those still running; a quarter within 7 and three quarters within 30. This study's 5 months is below that.

Counting every sponsor, half of 490 studies reached it within 17 months; a quarter within 9 and three quarters within 28.

How these figures are worked out, and which studies are left out of them.

Registered
Jan 3, 2024
First posted
Jan 29, 2024
StartActual
Nov 13, 2023
Primary completionActual
May 10, 2024
CompletionActual
Aug 7, 2024
Last update
Nov 22, 2024
What these dates mean
Start.
When the first participant enrolled; while estimated, when the study expects to open to recruitment.
Primary completion.
When the last participant was examined or treated to collect data for the study's main question, its primary outcome. A study that stopped early gives the date its data collection stopped.
Completion.
When the last participant's last visit took place, for every outcome, secondary ones and side effects included: the end of the study. Often later than primary completion, sometimes the same day.
Actual and Est.
Actual means the registry says the date has happened. Est. (estimated) is a date the sponsor plans or expects. It can still move, and the sponsor is expected to change it to actual once it has passed.
No changes since tracking began. The registry record was last updated on Nov 22, 2024, before this site started recording changes on Sep 25, 2026. Its history is on ClinicalTrials.gov ↗
10

Registry details

Key details

Study ID
NCT06229379
Responsible party
Haotian Lin (Professor, Sun Yat-sen University) — Principal investigator
First posted
Jan 29, 2024
Start date
Nov 13, 2023
Primary completion
May 10, 2024
Completion
Aug 7, 2024
Last update
Nov 22, 2024

Oversight

Data monitoring committee
Yes
FDA-regulated drug
No
FDA-regulated device
No
View the source record on ClinicalTrials.gov ↗

Not currently enrolling

This study is completed, as verified in Feb 2024. You cannot join it, but the record below documents what was studied.

Follow this study

Get an email when the registry record changes — status, dates, results — or when someone posts here.

Sign in to follow

Discussion

Questions and observations about this study, from anyone following it. Not medical advice, and not a channel to the study team — their contact details are on the registry record.

Sign in to join the discussion. Reading takes no account; posting does. You choose a display name, and a pseudonym is the default.

Nothing here yet. If you are running this trial, taking part in it, or weighing whether to, this is the place to say so.

Start the discussion