An observational study in Cancer, Respiratory Failure and Heart Diseases, sponsored by North Sichuan Medical College. Not yet recruiting at 1 site in China. Per ClinicalTrials.gov, last updated 2024-10-04.
Sponsored by North Sichuan Medical College · Observational
This retrospective clinical trial aims to better explore the potential of large language models in medicine by comparing the effectiveness of MDT consultations conducted by human doctors with those conducted by large language models.
The main questions to be addressed are:
Does using large language models to conduct anthropomorphic MDT consultations yield better results than using non-anthropomorphic processes? Is there a significant performance gap between MDT consultations conducted by large language models and those conducted by humans? How much greater is the economic benefit of MDT consultations from large language models compared to those conducted by humans?
Retrospectively collect MDT consultation records from the past 20 years in northern Sichuan in China, as well as anonymized patient medical records. Group 1: Different large language models are assigned to act as doctors from different departments and as MDT secretaries to summarize consultations. Group 2: The large language model directly outputs diagnostic and treatment recommendations for patients. Compare the outputs of groups 1 and 2 with human performance retrospectively, score them, and select the best model from each department for a re-evaluation through anthropomorphic MDT consultations, once again comparing them to human results.
1,650 studies on the registry are indexed under Respiratory Insufficiency; 296 are open to participants now.
This study's planned enrollment of 300 is above the median of 100 across 545 observational studies indexed under Respiratory Insufficiency.
Browse Respiratory Insufficiency studies →North Sichuan Medical College is the lead sponsor of 3 studies on the registry; 1 is open to participants now.
Counted across the registry records on this site, refreshed daily.
From hospital
Exclusion Criteria:
Using a locally deployed MedicalGPT, the commercially available online GPT-4o, Claude-3.5 Sonnet, GPT-4o mini, and Claude 3 Haiku, will each sequentially play the role of physicians from different departments involved in the Multi-Disciplinary Treatment Process. They will then sequentially take on the role of a summarizer to compile their recommendations into a final suggestion or treatment plan.
Diagnostic Test: GPT-4o · Diagnostic Test: GPT-4o mini · Diagnostic Test: MedicalGPT · Diagnostic Test: Claude-3.5 Sonnet · Diagnostic Test: Claude 3 Haiku
Using a locally deployed MedicalGPT, the commercial online GPT-4o, Claude-3.5 Sonnet, GPT-4o mini, and Claude 3 Haiku to output multidisciplinary consultation results in a single instance, without separately assuming roles for each department and then compiling the results.
Diagnostic Test: GPT-4o · Diagnostic Test: GPT-4o mini · Diagnostic Test: MedicalGPT · Diagnostic Test: Claude-3.5 Sonnet · Diagnostic Test: Claude 3 Haiku
In traditional multidisciplinary treatments, the results are documented in the consultation records of the patients involved, including the recommendations from doctors of various departments who participated in the consultation and the final summary by the secretary.
Diagnostic Test: Real Doctors
After scoring the results of the Anthropomorphized Process Large Language Model Multidisciplinary Treatment Group against the outcomes of the Real Doctors' Multi-Disciplinary Treatment Group on a department-by-department basis, the best substitute models and the best summary models for each department were selected. These top models are set to assume roles in a Multi-Disciplinary Treatment consultation.
Diagnostic Test: GPT-4o · Diagnostic Test: GPT-4o mini · Diagnostic Test: MedicalGPT · Diagnostic Test: Claude-3.5 Sonnet · Diagnostic Test: Claude 3 Haiku
Input all patient medical records, including text, examination reports, and imaging data, into GPT-4o. Use pre-tested prompts to establish department roles, enabling it to provide diagnostic and treatment recommendations pertinent to the respective department.
Input all patient medical records, including text, examination reports, and imaging data, into GPT-4o mini. Use pre-tested prompts to establish department roles, enabling it to provide diagnostic and treatment recommendations pertinent to the respective department.
Input all patient medical records, including text, examination reports, and imaging data, into MedicalGPT. Use pre-tested prompts to establish department roles, enabling it to provide diagnostic and treatment recommendations pertinent to the respective department.
Input all patient medical records, including text, examination reports, and imaging data, into Claude-3.5 Sonnet. Use pre-tested prompts to establish department roles, enabling it to provide diagnostic and treatment recommendations pertinent to the respective department.
Input all patient medical records, including text, examination reports, and imaging data, into Claude 3 Haiku. Use pre-tested prompts to establish department roles, enabling it to provide diagnostic and treatment recommendations pertinent to the respective department.
Retrospectively collect the diagnostic and treatment recommendations from the corresponding departments involved in the multidisciplinary treatment of past patients, as well as the overall recommendations.
Consultation Cost ($)
Time frame: From Multi-Disciplinary Treatment Process to Multi-Disciplinary Treatment Process until all json fields are output, the time taken by human doctors to record the time using His system generally does not exceed 12 hours.
Consultation Time (min)
Time frame: From Multi-Disciplinary Treatment Process to Multi-Disciplinary Treatment Process until all json fields are output, the time taken by human doctors to record the time using His system generally does not exceed 12 hours.
Comprehensiveness of the Multi-Disciplinary Treatment Results (Percentage Scale)
Time frame: From Multi-Disciplinary Treatment Process to Multi-Disciplinary Treatment Process until all json fields are output, the time taken by human doctors to record the time using His system generally does not exceed 12 hours.
Clarity of Multi-Disciplinary Treatment Results (Percentage Scale)
Time frame: From Multi-Disciplinary Treatment Process to Multi-Disciplinary Treatment Process until all json fields are output, the time taken by human doctors to record the time using His system generally does not exceed 12 hours.
Correctness of Multi-Disciplinary Treatment Results (Percentage Scale)
Time frame: From Multi-Disciplinary Treatment Process to Multi-Disciplinary Treatment Process until all json fields are output, the time taken by human doctors to record the time using His system generally does not exceed 12 hours.
Cross-Professional Team Collaboration Practice Assessment (CPAT)
Time frame: From Multi-Disciplinary Treatment Process to Multi-Disciplinary Treatment Process until all json fields are output, the time taken by human doctors to record the time using His system generally does not exceed 12 hours.
Rating Scale for Summarization
Time frame: From Multi-Disciplinary Treatment Process to Multi-Disciplinary Treatment Process until all json fields are output, the time taken by human doctors to record the time using His system generally does not exceed 12 hours.
Flesch-Kincaid Readability Test
Time frame: From Multi-Disciplinary Treatment Process to Multi-Disciplinary Treatment Process until all json fields are output, the time taken by human doctors to record the time using His system generally does not exceed 12 hours.
Ethical Compliance (Boolean)
Time frame: From Multi-Disciplinary Treatment Process to Multi-Disciplinary Treatment Process until all json fields are output, the time taken by human doctors to record the time using His system generally does not exceed 12 hours.
Plan to share: Yes
Supporting information: Study protocol, Sap, Icf, Csr, Analytic code
This study is not yet recruiting, as verified in Oct 2024. You cannot join it, but the record below documents what was studied.
Get an email when the registry record changes — status, dates, results — or when someone posts here.
Sign in to followQuestions and observations about this study, from anyone following it. Not medical advice, and not a channel to the study team — their contact details are on the registry record.
Sign in to join the discussion. Reading takes no account; posting does. You choose a display name, and a pseudonym is the default.
Nothing here yet. If you are running this trial, taking part in it, or weighing whether to, this is the place to say so.
North Sichuan Medical College