Artificial Intelligent Clinical Decision Support System Simulation Center Study for Technology Acceptance

NCT05816473 | Completed Results

• 2 other identifiers: 20000345211K23DK125718-01A1
• Study Type: interventional
• Target: 108
• Locations: 1 country
• Sites: 1

Brief Summary

The purpose of this research study is to measure the effect on of a large language model interface on the usability, attitudes, and provider trust when using a machine learning algorithm-based clinical decision support system in the setting of bleeding from the upper gastrointestinal tract (upper GIB). Specifically, the investigators are looking to assess the optimal implementation of such machine learning algorithms in simulation scenarios to best engender trust and improve usability. Participants will be randomized to either machine learning algorithm alone or algorithm with a large language model interface and exposed to simulation cases of upper GIB.

Conditions

Gastrointestinal Hemorrhage

Keywords

Implementation science; Artificial intelligence; Decision Support Systems, Clinical; Simulation Training

Outcome Measures

Primary Outcomes (1)

• Median Change in Attitudes Towards Machine Learning Algorithms in Clinical Care Using UTAUT

  The study will use a common set of dependent variables to assess baseline and post-intervention attitudes towards machine learning algorithms in clinical care using an adapted Unified Theory of Acceptance and Use of Technology (UTAUT) survey assessing perceived usefulness of the system, perceived ease of use, attitudes towards using it, behavioral intentions, and trust, measured with a 5-point Likert scale. Percent change in UTAUT survey response between Large Language Model-based Interaction and Machine Learning Dashboard at recruitment prior to administration of scenarios and immediately after completion of scenarios. The difference in time between the two will be approximately 60 minutes. Higher change indicates greater acceptance/intention to use the GutGPT+Dashboard.

  Approximately 60 minutes

Secondary Outcomes (1)

• Clinician Decision Making of Triage of GI Bleeding

  Approximately 60 minutes

Study Arms (2)

Large Language Model-based Interaction

EXPERIMENTAL

LLM-powered chatbot with the machine learning dashboard to provide the risk assessment and provide rationale based on interpretability metrics provided by the dashboard in which study participants can directly interact with using natural language. Participants will be provided the Generative Pre-trained Transformer (GPT) chatbot powered machine learning model dashboard.

Other: LLM

Machine Learning Dashboard

NO INTERVENTION

Machine learning algorithm output with an interactive dashboard that can be used to explain, or interpret the input factors that contribute most towards the generated risk score. Participants will have access to the machine learning dashboard only.

Interventions

LLM OTHER

Use of a Large Language Model (LLM) chatbot interface to Interact with the Machine Learning Algorithm and interpretability dashboard.

Large Language Model-based Interaction

Eligibility Criteria

• Sex: all
• Healthy Volunteers: Yes
• Age Groups: Child (0-17), Adult (18-64), Older Adult (65+)

You may qualify if:

• Internal Medicine residency trainees at study institution
• Emergency Medicine residency trainees at study institution

You may not qualify if:

• N/A

Sponsors & Collaborators

• Yale University: lead
• National Institute of Diabetes and Digestive and Kidney Diseases (NIDDK): collaborator

Study Sites (1)

Yale New Haven Hospital

New Haven, Connecticut, 06510, United States

Location

Related Publications (5)

• Laine L. Risk Assessment Tools for Gastrointestinal Bleeding. Clin Gastroenterol Hepatol. 2016 Nov;14(11):1571-1573. doi: 10.1016/j.cgh.2016.08.003. Epub 2016 Aug 10. No abstract available.

  PMID: 27521511 BACKGROUND

• Laine L, Jensen DM. Management of patients with ulcer bleeding. Am J Gastroenterol. 2012 Mar;107(3):345-60; quiz 361. doi: 10.1038/ajg.2011.480. Epub 2012 Feb 7.

  PMID: 22310222 BACKGROUND

• Leonardi, P. M. 2009. Why do people reject new technologies and stymie organizational changes of which they are in favor? Exploring misalignments between social interactions and materiality. Human Communication Research, 35(3): 407-441.

  BACKGROUND

• Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003). User Acceptance of Information Technology: Toward a Unified View. MIS Quarterly, 27(3), 425-478

  BACKGROUND

• Chung S, Giuffre M, Rajashekar N, Pu Y, Shin YE, Kresevic S, Chan C, Nakamura-Sakai S, You K, Saarinen T, Hsiao A, Wong AH, Evans L, McCall T, Kizilcec RF, Sekhon J, Laine L, Shung DL. Usability and adoption in a randomized trial of GutGPT a GenAI tool for gastrointestinal bleeding. NPJ Digit Med. 2025 Aug 18;8(1):527. doi: 10.1038/s41746-025-01896-5.

  PMID: 40825997 DERIVED

MeSH Terms

Conditions

Gastrointestinal Hemorrhage

Condition Hierarchy (Ancestors)

Gastrointestinal Diseases; Digestive System Diseases; Hemorrhage; Pathologic Processes; Pathological Conditions, Signs and Symptoms

Results Point of Contact

• Title: Sunny Chung
• Organization: Yale School of Medicine

Sunny.chung@yale.edu

203-824-1459

Study Officials

• Dennis Shung, MD

  Yale School of Medicine Section of Digestive Diseases

  PRINCIPAL INVESTIGATOR

Publication Agreements

• PI is Sponsor Employee: Yes

Study Design

• Study Type: interventional
• Phase: not applicable
• Allocation: NA
• Masking: NONE
• Purpose: HEALTH SERVICES RESEARCH
• Intervention Model: PARALLEL
• Sponsor Type: OTHER
• Responsible Party: SPONSOR

Study Record Dates

• First Submitted: March 13, 2023
• First Posted: April 18, 2023
• Study Start: May 23, 2023
• Primary Completion: December 31, 2024
• Study Completion: December 31, 2024
• Last Updated: May 22, 2026
• Results First Posted: May 22, 2026

Record last verified: 2026-05

Data Sharing

• IPD Sharing: Will not share

Locations

US (1)