ClinicalTrials.gg
CompletedNCT07933107Updated Nov 18, 2026

Evaluating Large Language Models in Nursing Care Decision-Making

An interventional study of Independent planning and LLM-assisted planning in Simulation-based Nursing Care Planning, sponsored by Chimei Medical Center. Completed at 1 site in Taiwan. Open to participants aged 18 Years and older, including healthy volunteers. Per ClinicalTrials.gov, last updated 2026-11-18.

Sponsored by Chimei Medical Center · Not applicable, Interventional, and Health services research

How we know

From the registry’s dates

  • Registered 5 months after the study started (first participant enrolled May 2026, registered Nov 2026).
Updated Nov 18, 2026Start date movedPrimary outcomes revised+2 moreGo to Updates ↓

Help build ClinicalTrials.gg — short surveys, remote interviews or testing, in your language. Take part →

Phase
Not applicable
Study type
Interventional
Enrollment
120
Allocation
Randomized
Ages
18 Years and older
Sex
All
01

Study summary

This study evaluated whether assistance from a large language model (LLM), a type of artificial intelligence that generates text, improved the quality of nursing care plans and reduced the time needed to complete them.

The study included 120 registered nurses working in internal medicine wards, surgical wards, and intensive care units at a medical center in southern Taiwan. Nurses were randomly assigned to one of two groups:

  • Independent planning: Nurses prepared, reviewed, and submitted a care plan without LLM assistance.
  • LLM-assisted planning: Nurses entered prompts to obtain an LLM-generated draft, then reviewed and revised the draft before submitting their final care plan.

Each nurse completed only the assigned workflow during a single study session. Participants completed a brief baseline questionnaire and prepared nursing problems, goals, and interventions for a standardized simulated clinical case. The diagnoses in the simulated cases were included for scenario purposes only; no patients were enrolled. Tasks were performed in a secure hospital test environment without real patient data.

Expert nurse raters who were not informed of group assignments independently evaluated the final care plans. The nursing care decision quality assessment had a maximum total score of 30 points, comprising decision-content quality (25 points) and decision-language quality (5 points). The system recorded task completion time.

Researchers compared the groups' final care-plan quality scores and task completion times.

Read the detailed description

Nursing care planning requires nurses to identify nursing problems, establish appropriate goals, and select feasible interventions. Large language models (LLMs) can generate draft text, but their contribution to nursing care-plan quality and task efficiency requires evaluation. In this study, nurses were responsible for reviewing the generated content and determining the final care plan.

Study design and setting

This single-center, parallel-group randomized controlled study compared Independent planning and LLM-assisted planning using standardized simulated clinical cases. The study was conducted in internal medicine wards, surgical wards, and intensive care units at a medical center in southern Taiwan.

Among 652 nurses who met the eligibility criteria and expressed willingness to participate, 120 were selected through stratified random sampling, with 40 nurses from each clinical setting. Within each setting, participants were individually randomized in a 1:1 ratio to Independent planning or LLM-assisted planning. Each group included 60 participants, comprising 20 nurses from each clinical setting. Excel-generated random-number procedures were used for sampling and allocation.

Each participant completed one assigned workflow during a single study session. The study used a posttest-only assessment of the submitted care plan; participants did not complete both study conditions.

Study procedures

Participants completed a brief baseline questionnaire and a care-planning task based on a standardized simulated clinical case relevant to their clinical setting. The standardized cases included a pneumonia case for the internal medicine setting, a cholelithiasis case for the surgical setting, and a pneumonia case for the intensive care setting. These diagnoses were included solely as features of the simulated scenarios; no patients were enrolled or treated. The study evaluated nurses' care-planning performance, not patient outcomes. All tasks were performed in a secure hospital test environment separated from routine clinical systems. No real patient data were used, and no audio or video recordings were made.

In the Independent planning group, participants independently prepared nursing problems, goals, and interventions, reviewed their work, and submitted a final care plan without LLM assistance.

In the LLM-assisted planning group, participants entered prompts based on the simulated case to obtain an LLM-generated draft. They reviewed the draft, corrected or removed inappropriate content, added information as needed, and submitted a final care plan. Participants retained responsibility for the final nursing problems, goals, and interventions.

Outcome assessment

The primary outcomes were expert-rated nursing care decision quality and total task completion time.

The nursing care decision quality assessment had a maximum total score of 30 points, comprising decision-content quality, with a maximum of 25 points, and decision-language quality, with a maximum of 5 points. Expert nurse raters independently evaluated the final submitted care plans without being informed of participants' group assignments. Higher scores indicated better assessed quality. Inter-rater reliability was assessed.

Task completion time was calculated from system-recorded task-start and final-submission timestamps. The same timing boundaries applied to both groups. For LLM-assisted planning, the task interval included prompt entry, LLM processing, and participant review and revision.

Secondary outcomes were textual overlap and semantic similarity between each participant's final care plan and the expert consensus reference for the same simulated case, assessed using ROUGE-L and BERTScore, respectively. These measures were calculated offline from the submitted text.

The protocol also specified AI-only outputs as supplementary algorithmic benchmarks. These outputs were separate from the randomized comparison and did not constitute an additional participant group or contribute to enrollment.

Data protection

Study questionnaires, care-plan texts, ratings, and workflow timestamps were linked using study codes. Research data were stored on secure institutional systems with access restricted to authorized study personnel. Identifiable information was not included in the simulated cases submitted to the LLM. Data retention and destruction followed the approved study protocol and institutional requirements.

02

Conditions studied

  • Simulation-based Nursing Care Planning

Keywords

  • Nursing care decision-making
  • Large language models
  • Artificial intelligence
  • Human-AI collaboration
  • Clinical simulation
  • Nursing informatics
  • Acute care nursing
03

In context

Lead sponsor

Chimei Medical Center is the lead sponsor of 28 studies on the registry; 6 are open to participants now.

Counted across the registry records on this site, refreshed daily.

04

Who can participate

Ages eligible
18 Years and older
Sexes eligible
All
Accepts healthy volunteers
Yes

Go through the criteria one by one — yes, no or not sure — and take the list to your doctor or the study team. Kept in this browser only.

Inclusion criteria

  • Registered nurses employed in internal medicine wards, surgical wards, or intensive care units at the participating medical center.
  • Aged 18 years or older.
  • Willing to participate voluntarily and provide written .

Exclusion criteria

Exclusion Criteria:

  • Nurses holding a deputy head nurse position or a higher nursing management position.
  • Nurse practitioners.
  • Student nurses.
  • Nurses who have not directly engaged in nursing care decision-making during the three weeks preceding enrollment.
Terms in these criteria, in plain language(1)

Underlined terms in the criteria open these too. Explained by clinicaltrials.gg, not part of the registry record. The study team decides who can take part.

Informed consent

Before you join, the study team explains what taking part involves — its purpose, what happens, the risks and possible benefits — and answers your questions, so you can decide freely. Consent is usually recorded on a form (the informed consent form, ICF) signed by you or, where the law allows, by someone legally authorized to decide for you.

You can keep asking questions during the study, you are told of new information that matters, and you can choose to leave at any time.

05

Study design

Phase
Not applicable
Primary purpose
Health services research
Allocation
Randomized
Intervention model
Parallel assignment
Masking
Single (Outcomes assessor)
Enrollment
120 participants (actual)

Study arms

  • Active comparator
    Independent planning

    Participants independently prepare a nursing care plan for a standardized simulated clinical case, including nursing problems, goals, and interventions. They review and revise their own work before submitting the final care plan. No LLM assistance or AI-generated suggestions are used.

    Other: Independent planning

  • Experimental
    LLM-assisted planning

    Participants prepare a nursing care plan for a standardized simulated clinical case with LLM assistance. They enter prompts to obtain an LLM-generated draft, review and revise the draft, and submit the final nursing problems, goals, and interventions. Participants remain responsible for the content of the final care plan.

    Other: LLM-assisted planning

Interventions

  • OtherIndependent planning

    A single-session care-planning task performed in a secure hospital test environment. Participants review the simulated case and independently formulate, review, and submit nursing problems, goals, and interventions using their nursing knowledge and judgment, without LLM assistance.

  • OtherLLM-assisted planning

    A single-session care-planning task performed in a secure hospital test environment. Participants enter prompts based on the simulated case to obtain an LLM-generated draft of nursing problems, goals, and interventions. They evaluate the draft, correct or remove inappropriate content, add information as needed, and submit a final care plan. Only simulated case information is used; no real patient data are entered.

06

What researchers measure

Primary outcomes

  1. Expert-rated nursing care decision quality (six-item total score)

    The final nursing care plan for a standardized simulated clinical case is independently assessed by three expert nurse raters blinded to group assignment, using a study-specific six-item rating instrument. Five items assess decision-content quality and one item assesses decision-language quality. Each item is scored from 1 to 5. The content subtotal ranges from 5 to 25, and the language score ranges from 1 to 5. For each rater, the six item scores are summed to obtain a total score ranging from 6 to 30. The participant-level score is the arithmetic mean of the three raters' total scores. Higher scores indicate better assessed nursing care decision quality.

    Time frame: Posttest (at the end of the single simulation session; one assessment).

  2. Total care-planning task completion time (seconds)

    Total elapsed time, in seconds, from the start of the simulated care-planning task to submission of the final care plan, calculated from system-recorded start and submission timestamps. The same timing boundaries apply to both groups. Time includes case review, drafting, review, and revision. In the LLM-assisted planning group, it also includes prompt entry and LLM processing within the task interval. Lower values indicate shorter task completion time.

    Time frame: During the single simulation session, from task start to final submission.

Secondary outcomes

  1. Textual overlap with an expert consensus reference (ROUGE-L)

    ROUGE-L score comparing each participant's final submitted care plan with the expert consensus reference for the same simulated clinical case. The score quantifies textual overlap based on the longest common subsequence between the submitted text and the reference. Higher scores indicate greater textual overlap. Scores are calculated offline from the final care plans submitted after the assigned workflow.

    Time frame: Posttest (based on the final care plan submitted at the end of the single simulation session; calculated offline).

  2. Semantic similarity to an expert consensus reference (BERTScore)

    BERTScore comparing each participant's final submitted care plan with the expert consensus reference for the same simulated clinical case. The score assesses semantic similarity using contextual text embeddings. Higher scores indicate greater semantic similarity under the specified model and calculation settings. Scores are calculated offline from the final care plans submitted after the assigned workflow.

    Time frame: Posttest (based on the final care plan submitted at the end of the single simulation session; calculated offline).

07

Study locations

1 site
  • Chi Mei Medical Center
    Tainan, 710402, Taiwan
08

References and documents

Individual participant data

Plan to share: No

No publications or documents are linked to this record.

09

Timeline

2 registry updates since Sep 25, 2026
ElapsedEstimatedEstimated, not yet reachedDate before it movedWhere the date was before it moved

Ran from May 2026 to primary completion in Jul 2026, 1 month.

Start moved once since we started tracking on Sep 25, 2026: Mar 2026 → May 2026, on Nov 18, 2026.

Earlier changes are on the record history on ClinicalTrials.gov ↗

How these figures are worked out, and which studies are left out of them.

Registered
Nov 2, 2026
First posted
Nov 12, 2026
StartActual
May 14, 2026was Mar 2026
Primary completionActual
Jul 9, 2026
CompletionActual
Jul 9, 2026
Last update
Nov 18, 2026
What these dates mean
Start.
When the first participant enrolled; while estimated, when the study expects to open to recruitment.
Primary completion.
When the last participant was examined or treated to collect data for the study's main question, its primary outcome. A study that stopped early gives the date its data collection stopped.
Completion.
When the last participant's last visit took place, for every outcome, secondary ones and side effects included: the end of the study. Often later than primary completion, sometimes the same day.
Actual and Est.
Actual means the registry says the date has happened. Est. (estimated) is a date the sponsor plans or expects. It can still move, and the sponsor is expected to change it to actual once it has passed.
Start date
Mar 14, 2026→May 14, 2026
Nov 18, 2026
Primary outcomes
2 time frame changed
What changed in the outcomes
  • Time frameExpert-rated nursing care decision quality (six-item total score)Day 1 (at the end of the single simulation session). → Posttest (at the end of the single simulation session; one assessment).
  • Time frameTotal care-planning task completion time (seconds)Day 1 (during the single simulation session). → During the single simulation session, from task start to final submission.
Nov 18, 2026
Secondary outcomes
2 time frame changed
What changed in the outcomes
  • Time frameTextual overlap with an expert consensus reference (ROUGE-L)Day 1 (final care plan submitted at the end of the single simulation session; scored offline). → Posttest (based on the final care plan submitted at the end of the single simulation session; calculated offline).
  • Time frameSemantic similarity to an expert consensus reference (BERTScore)Day 1 (final care plan submitted at the end of the single simulation session; scored offline). → Posttest (based on the final care plan submitted at the end of the single simulation session; calculated offline).
Nov 18, 2026
Title
Evaluating Large Language Models in Nursing Care Decision-Making: A Randomized Controlled Trial
Nov 18, 2026
Registered
First appeared on the registry
Nov 12, 2026
Show all 2 updates
  1. Nov 18, 2026
    Start date Mar 14, 2026→May 14, 2026
    Primary outcomes 2 time frame changed
    What changed in the outcomes
    • Time frameExpert-rated nursing care decision quality (six-item total score)Day 1 (at the end of the single simulation session). → Posttest (at the end of the single simulation session; one assessment).
    • Time frameTotal care-planning task completion time (seconds)Day 1 (during the single simulation session). → During the single simulation session, from task start to final submission.
    Secondary outcomes 2 time frame changed
    What changed in the outcomes
    • Time frameTextual overlap with an expert consensus reference (ROUGE-L)Day 1 (final care plan submitted at the end of the single simulation session; scored offline). → Posttest (based on the final care plan submitted at the end of the single simulation session; calculated offline).
    • Time frameSemantic similarity to an expert consensus reference (BERTScore)Day 1 (final care plan submitted at the end of the single simulation session; scored offline). → Posttest (based on the final care plan submitted at the end of the single simulation session; calculated offline).
    Title Evaluating Large Language Models in Nursing Care Decision-Making: A Randomized Controlled Trial
    + 2 other changes: description and conditions
  2. Nov 12, 2026
    First appeared on the registry

From the registry record's own update history. This site started tracking changes on Sep 25, 2026; for anything earlier, see the record history on ClinicalTrials.gov ↗

10

Registry details

Key details

Study ID
NCT07933107
Lead sponsor
Chimei Medical Center
Responsible party
Sponsor
First posted
Nov 12, 2026
Start date
May 14, 2026
Primary completion
Jul 9, 2026
Completion
Jul 9, 2026
Last update
Nov 18, 2026

Oversight

Data monitoring committee
No
FDA-regulated drug
No
FDA-regulated device
No
View the source record on ClinicalTrials.gov ↗

Not currently enrolling

This study is completed, as verified in Nov 2026. You cannot join it, but the record below documents what was studied.

Follow this study

Get an email when the registry record changes — status, dates, results — or when someone posts here.

Sign in to follow

Discussion

Questions and observations about this study, from anyone following it. Not medical advice, and not a channel to the study team — their contact details are on the registry record.

Sign in to join the discussion. Reading takes no account; posting does. You choose a display name, and a pseudonym is the default.

Nothing here yet. If you are running this trial, taking part in it, or weighing whether to, this is the place to say so.

Start the discussion