Loading search results

Outcome Measures

Primary Outcomes (2)

Expert-rated nursing care decision quality (six-item total score)
The final nursing care plan for a standardized simulated clinical case is independently assessed by three expert nurse raters blinded to group assignment, using a study-specific six-item rating instrument. Five items assess decision-content quality and one item assesses decision-language quality. Each item is scored from 1 to 5. The content subtotal ranges from 5 to 25, and the language score ranges from 1 to 5. For each rater, the six item scores are summed to obtain a total score ranging from 6 to 30. The participant-level score is the arithmetic mean of the three raters' total scores. Higher scores indicate better assessed nursing care decision quality.
Day 1 (at the end of the single simulation session).
Total care-planning task completion time (seconds)
Total elapsed time, in seconds, from the start of the simulated care-planning task to submission of the final care plan, calculated from system-recorded start and submission timestamps. The same timing boundaries apply to both groups. Time includes case review, drafting, review, and revision. In the LLM-assisted planning group, it also includes prompt entry and LLM processing within the task interval. Lower values indicate shorter task completion time.
Day 1 (during the single simulation session).

Secondary Outcomes (2)

Textual overlap with an expert consensus reference (ROUGE-L)
Day 1 (final care plan submitted at the end of the single simulation session; scored offline).
Semantic similarity to an expert consensus reference (BERTScore)
Day 1 (final care plan submitted at the end of the single simulation session; scored offline).

Study Arms (2)

Independent planning

ACTIVE COMPARATOR

Participants independently prepare a nursing care plan for a standardized simulated clinical case, including nursing problems, goals, and interventions. They review and revise their own work before submitting the final care plan. No LLM assistance or AI-generated suggestions are used.

Other: Independent planning

LLM-assisted planning

EXPERIMENTAL

Participants prepare a nursing care plan for a standardized simulated clinical case with LLM assistance. They enter prompts to obtain an LLM-generated draft, review and revise the draft, and submit the final nursing problems, goals, and interventions. Participants remain responsible for the content of the final care plan.

Other: LLM-assisted planning

Interventions

Independent planningOther

A single-session care-planning task performed in a secure hospital test environment. Participants review the simulated case and independently formulate, review, and submit nursing problems, goals, and interventions using their nursing knowledge and judgment, without LLM assistance.

Independent planning

LLM-assisted planningOther

A single-session care-planning task performed in a secure hospital test environment. Participants enter prompts based on the simulated case to obtain an LLM-generated draft of nursing problems, goals, and interventions. They evaluate the draft, correct or remove inappropriate content, add information as needed, and submit a final care plan. Only simulated case information is used; no real patient data are entered.

Study Design

Study Type

interventional

Phase

not applicable

Allocation

RANDOMIZED

Masking

SINGLE

Who Masked

OUTCOMES ASSESSOR

Masking Details

Expert nurse raters independently assess study-coded final care plans without being informed of participants' group assignments. Group labels are not provided to the raters. Participants and personnel administering the study workflows are not masked because LLM use is apparent during task performance.

Purpose

HEALTH SERVICES RESEARCH

Intervention Model

PARALLEL

Model Details: Single-center, parallel-group randomized trial comparing Independent planning with LLM-assisted planning in a single simulation session. A stratified random sample of 120 nurses was selected, with 40 from each of three clinical settings: internal medicine wards, surgical wards, and intensive care units. Within each setting, participants were individually randomized 1:1, yielding 60 participants per group and 20 per setting in each group. Each participant completed only the assigned workflow, followed by assessment of the final submitted care plan. AI-only benchmark outputs are separate from the randomized comparison and are not counted as participants or as a third study arm.

Sponsor Type

OTHER

Responsible Party

SPONSOR

Evaluating Large Language Models in Nursing Care Decision-Making: A Randomized Controlled Trial

Brief Summary

Registry Overview

Trial Relationships

Study Timeline

Study Start

Primary Completion

Study Completion

First Submitted

First Posted

Conditions

Keywords

Outcome Measures

Primary Outcomes (2)

Secondary Outcomes (2)

Study Arms (2)

Independent planning

LLM-assisted planning

Interventions

Eligibility Criteria

You may qualify if:

Sponsors & Collaborators

Study Sites (1)

Enrollment

Study Design

Study Record Dates

You may not qualify if:

Data Sharing

Source record

Locations

Brief Summary

Registry Overview

Trial Relationships

Related Scientific Literature

Study Timeline

Study Start

Primary Completion

Study Completion

First Submitted

First Posted

Conditions

Keywords

Outcome Measures

Primary Outcomes (2)

Secondary Outcomes (2)

Study Arms (2)

Independent planning

LLM-assisted planning

Interventions

Eligibility Criteria

You may qualify if:

Sponsors & Collaborators

Study Sites (1)

Enrollment

Study Design

Study Record Dates

You may not qualify if:

Data Sharing

Source record

Locations