The Accuracy and Efficacy of Large Language Model Written Hospital Course Summaries

NCT07491068 | Not Yet Recruiting

• 1 other identifier: 2IK2026_1
• Study Type: interventional
• Target: 786
• Locations: 1 country
• Sites: 1

Brief Summary

Background: Physicians worldwide face an increasing administrative burden that diverts time from direct patient care. Among inpatient documentation tasks, authoring hospital course summaries is particularly time-consuming and critical for safe care transitions. Large language models (LLMs) have shown promise for clinical text generation; however, robust evidence from randomized, evaluator-blinded trials conducted in routine hospital practice remains limited. Objectives: The CLEAN study aims to evaluate whether LLM-assisted, specialistedited generation of hospital course summaries is non-inferior in safety compared with standard clinician-written documentation in routine inpatient care. Secondary objectives include noninferiority assessments of resident-edited and unedited LLMgenerated summaries. Additional objectives are to evaluate summary quality across predefined domains, quantify physician documentation time, assess LLM generation stability, measure clinician adoption following the randomized phase, and examine inter-, intra-observer, and test-retest reliability of expert assessments. Methods: This is a single-centre, double-campus, exploratory randomized controlled non-inferiority trial conducted at a tertiary university hospital. Consecutive hospital discharges across multiple clinical departments are randomized 1:1 to either an LLM-assisted documentation workflow or standard manual authorship. The intervention integrates an on-premise LLM into a parallel hospital information system, generating draft hospital course summaries from complete, uncurated clinical documentation, which physicians may review and edit prior to finalization. Safety, the primary outcome, defined as presence of all important information and absence of incorrect/hallucinated information, is assessed by an adjudication committee blinded to documentation workflow. Secondary outcomes include content validity, workflow efficiency, generation stability, post-trial clinician adoption, and reliability metrics. A total of 786 discharge episodes are required to assess non-inferiority using a predefined margin of 5 percentage points. Ethics and Dissemination: The study will be conducted in accordance with the Declaration of Helsinki, Good Clinical Practice, and the General Data Protection Regulation. A waiver of informed consent is sought due to minimal risk and exclusive use of routine clinical data. Results will be disseminated through peer-reviewed publication and engagement with healthcare stakeholders.

Conditions

Hospitalizations

Keywords

generative large language models; automated discharge summary; AI; documentation; Electronic health records

Outcome Measures

Primary Outcomes (1)

• Safety assessed by outcome evaluator on an ordinal scale (1,2,3)

  Safety will be assessed using an ordinal scale ranging from "acceptable without further changes", through "acceptable with minor revisions", to not "unacceptable in its current form" ( a combination of either absence of important information or presence of incorrect/hallucinated information)

  Assessment at one time point - hospital discharge (up to 5 days)

Secondary Outcomes (8)

• Safety of the edited and unedited LLM generated course summaries evaluated on an ordinal scale (1,2,3)

  At hospital discharge (up to 5 days)

• Secondary outcome: Content validity across the following predefined domains (5C).

  At hospital discharge (up to 5 days)

• Time to complete hospital summary (seconds)

  At hospital discharge (up to 5 days)

• Generation stability assessed on an ordinal scale (1,2,3)

  From hospital discharge to 30 days after hospital discharge

• Adoption. Percentage of hospital discharges where the LLM generated course summary was utilized.

  From hospital discharge to 30 days after hospital discharge

Study Arms (2)

LLM assisted

EXPERIMENTAL

The intervention consists of an LLM assisted workflow for generating the hospital course summary at discharge. The treating physician initiates discharge using an application - CorteVision Hospital Suite - connected to the hospital Informix database. The output of the model - generated draft is returned to the application interface, where the treating physician reviews and may edit, correct, expand, or shorten the text before finalization. The finalized hospital course summary is entered into the medical record only after physician review and confirmation.

Device: LLM assisted workflow for generating the hospital course summary

Control

NO INTERVENTION

Standard, manual generation of hospital course summary manually by responsible physician.

Interventions

LLM assisted workflow for generating the hospital course summary DEVICE

The intervention consists of an LLM assisted workflow for generating the hospital course summary at discharge. The treating physician initiates discharge using an application - CorteVision Hospital Suite - connected to the hospital Informix database. The output of the model - generated draft is returned to the application interface, where the treating physician reviews and may edit, correct, expand, or shorten the text before finalization. The finalized hospital course summary is entered into the medical record only after physician review and confirmation.

LLM assisted

Eligibility Criteria

• Age: 18 Years+
• Sex: all
• Healthy Volunteers: No
• Age Groups: Adult (18-64), Older Adult (65+)

Sponsors & Collaborators

• Pavol Jozef Safarik University: lead
• Louis Pasteur University Hospital, Košice: collaborator

Study Sites (1)

Louis Pasteur University Hospital Kosice

Košice, Košice Region, 04001, Slovakia

Location

Related Publications (3)

• Ganzinger M, Kunz N, Fuchs P, Lyu CK, Loos M, Dugas M, Pausch TM. Automated generation of discharge summaries: leveraging large language models with clinical data. Sci Rep. 2025 May 12;15(1):16466. doi: 10.1038/s41598-025-01618-7.

  PMID: 40355506 BACKGROUND

• Kripalani S, LeFevre F, Phillips CO, Williams MV, Basaviah P, Baker DW. Deficits in communication and information transfer between hospital-based and primary care physicians: implications for patient safety and continuity of care. JAMA. 2007 Feb 28;297(8):831-41. doi: 10.1001/jama.297.8.831.

  PMID: 17327525 BACKGROUND

• Burden M, Astik G, Auerbach A, Bowling G, Kangelaris KN, Keniston A, Kochar A, Leykum LK, Linker AS, Sakumoto M, Rogers K, Schwatka N, Westergaard S. Identifying and Measuring Administrative Harms Experienced by Hospitalists and Administrative Leaders. JAMA Intern Med. 2024 Sep 1;184(9):1014-1023. doi: 10.1001/jamainternmed.2024.1890.

  PMID: 38913371 BACKGROUND

Study Officials

• Jakub Gazda, MD, PhD

  Pavol Jozef Safarik University

  PRINCIPAL INVESTIGATOR

Central Study Contacts

Jakub Gazda, MD, PhD

CONTACT

+421556403517; jakub.gazda@upjs.sk

Martin Janicko, MD, PhD

CONTACT

+421556403527; martin.janicko@upjs.sk

Study Design

• Study Type: interventional
• Phase: not applicable
• Allocation: RANDOMIZED
• Masking: SINGLE
• Who Masked: OUTCOMES ASSESSOR
• Purpose: HEALTH SERVICES RESEARCH
• Intervention Model: PARALLEL
• Sponsor Type: OTHER
• Responsible Party: PRINCIPAL INVESTIGATOR
• PI Title: Associate professor

Study Record Dates

• First Submitted: February 25, 2026
• First Posted: March 24, 2026
• Study Start: June 1, 2026
• Primary Completion (Estimated): December 31, 2026
• Study Completion (Estimated): December 31, 2027
• Last Updated: March 30, 2026

Record last verified: 2026-03

Locations

SL (1)