Effect of Agent-assisted, LLM-assisted and Traditional Workflows on Physician Admission Diagnosis

NCT07760051 | Not Yet Recruiting

• 1 other identifier: 2026-0977
• Study Type: interventional
• Target: 180
• Locations: 1 country
• Sites: 1

Brief Summary

The goal of this clinical trial is to evaluate whether AI-assisted workflows improve physicians' admission diagnosis performance on standardized simulated inpatient cases, among practicing internal medicine and surgery physicians across all seniority levels and across three tiers of the Chinese healthcare system. The main questions it aims to answer are:

• Does the Agent-assisted workflow yield better structured admission diagnosis scores than standalone LLM assistance?
• Does the Agent-assisted workflow outperform the traditional workflow without AI tools? Researchers will compare three parallel groups (traditional workflow group, LLM-only group, Agent group) to determine whether the Agent tool can improve diagnostic accuracy and efficiency. Participants will:
• Be recruited from 15 hospitals in China and participate remotely under video proctoring
• Be randomly assigned to one of the three fixed workflows, with randomization stratified by hospital tier, specialty and seniority
• Complete 6 anonymized simulated HIS admission cases within one hour
• Submit structured answers for each case covering principal diagnosis, secondary diagnoses, differential diagnoses, diagnostic justification, next diagnostic or therapeutic steps, consultation and referral decisions, and diagnostic confidence
• Have their operation logs and time consumption recorded automatically by the study platform

Conditions

Clinical Decision Support Systems; Diagnostic Reasoning; Artificial Intelligence (AI)

Keywords

large language model; AI agent; clinical decision support; admission diagnosis; diagnostic accuracy; diagnostic reasoning; randomized controlled trial

Outcome Measures

Primary Outcomes (1)

• Mean Normalized Structured Diagnostic Reasoning Score

  Mean of the rescaled case scores (each case rescaled to 100), divided by the number of cases completed; range 0 to 100.

  Within one-hour study

Secondary Outcomes (3)

• Degree of Adherence to AI-Generated Recommendations

  Within one-hour study

• Active Response Time per Case

  Within one-hour study

• Normalized Score for Next Diagnostic or Therapeutic Steps

  Within one-hour study

Study Arms (3)

Agent group

EXPERIMENTAL

Other: Agent Workflow

LLM group

EXPERIMENTAL

Other: LLM Workflow

traditional group

ACTIVE COMPARATOR

Other: traditional workflow

Interventions

Agent Workflow OTHER

Conventional resources (pre-admission clinical record, search engines) plus an in-system Agent entry that automatically reads the full record and report images, produces a structured summary with source-text tracing, and supports multi-turn Q\&A and one-click editable drafts.

Agent group

LLM Workflow OTHER

Conventional resources plus an in-system multi-turn AI dialogue entry. The AI does not automatically read the record; participants paste text or send partial screenshots.

LLM group

traditional workflow OTHER

Conventional resources only: the pre-admission clinical record, standard search engines. No AI assistance.

traditional group

Eligibility Criteria

• Age: 18 Years+
• Sex: all
• Healthy Volunteers: Yes
• Age Groups: Adult (18-64), Older Adult (65+)

You may qualify if:

• Hold a Medical Practitioner Qualification Certificate and/or Medical License, or be a recognized standardized resident physician; able to independently read electronic medical records, laboratory and imaging reports on an HIS.
• Currently engaged in clinical work in internal medicine or surgery at one of the 15 participating hospitals.
• Able to complete the 1-hour case assessment and the preceding training module and qualification test.
• Able to participate remotely under video proctoring, with a stable internet connection and a working camera.
• Voluntarily agree to participate and sign the informed consent form, including the declaration not to use unauthorized AI tools during the assessment.
• Have not participated in case drafting, review, rubric development, or any activity that may leak the reference standard.
• Stratified enrollment by hospital tier (provincial tertiary, municipal, primary care), specialty (internal medicine, surgery) and seniority (senior = Associate Chief Physician or above; junior-to-mid-level = below Associate Chief Physician, prioritizing residents, attending physicians and those with ≤ 8 years of practice).

You may not qualify if:

• Have previously accessed the official test cases or reference standard of this study.
• Unable to complete the training module, qualification test, or all experimental tasks.
• Have conflicts of interest, e.g. participation in developing core algorithms of the tested system.
• Unwilling to comply with remote proctoring, including keeping the camera on throughout.
• Violate study rules during the assessment (unauthorized AI tools, discussing cases, copying answers).
• Judged unsuitable by the investigators.

Sponsors & Collaborators

• Second Affiliated Hospital, Zhejiang University, School of Medicine: lead
• Shangrao People's Hospital: collaborator

Study Sites (1)

2nd Affiliated Hospital, School of Medicine, Zhejiang University

Hangzhou, Zhejiang, 310009, China

Location

Study Officials

• Yuan Ding

  Second Affiliated Hospital, Zhejiang University, School of Medicine

  STUDY CHAIR

Central Study Contacts

Yixin Zhang

CONTACT

+86 19157950225; 12518552@zju.edu.cn

Study Design

• Study Type: interventional
• Phase: not applicable
• Allocation: RANDOMIZED
• Masking: SINGLE
• Who Masked: OUTCOMES ASSESSOR
• Masking Details: Participants and investigators cannot be masked, as the intervention is the workflow itself. Outcome assessors scoring the structured answers are blinded to group allocation; submitted answers are anonymized and stripped of workflow-identifying information before scoring.
• Purpose: HEALTH SERVICES RESEARCH
• Intervention Model: PARALLEL
• Sponsor Type: OTHER
• Responsible Party: SPONSOR

Model Details: Three-arm parallel-group design with 1:1:1 allocation. All arms share the same conventional resources (pre-admission clinical record, search engines); the arms differ only in the AI entry added on top (none / LLM / Agent). Randomization is stratified by hospital tier, specialty and seniority.

Study Record Dates

• First Submitted: July 29, 2026
• First Posted: August 12, 2026
• Study Start (Estimated): September 1, 2026
• Primary Completion (Estimated): December 1, 2026
• Study Completion (Estimated): December 1, 2026
• Last Updated: August 12, 2026

Record last verified: 2026-08

Data Sharing

• IPD Sharing: Will not share

Locations

CN (1)