DHL Survey on Generative AI for MyChart Messaging

NCT06108037 | Completed DMC

• 1 other identifier: Pro00113587
• Study Type: interventional
• Target: 1,454
• Locations: 1 country
• Sites: 1

Brief Summary

The purpose of this study is to understand how patients feel about the use of computer programs to create responses when they send electronic messages to their doctors.

Conditions

Communication

Keywords

Artificial Intelligence; Generative Artificial Intelligence

Outcome Measures

Primary Outcomes (3)

• Patient satisfaction, as measured by survey

  Likert-scale responses to satisfaction question: "I am satisfied with this interaction", on a scale from 1-5 with answer options of Strongly Disagree (1), Disagree (2), Neither agree nor disagree (3), Agree (4), and Strongly agree (5).

  Up to 2 weeks

• Patient attitudes towards utility, as measured by survey

  Likert-scale responses to utility question: "The information is useful", on a scale from 1-5 with answer options of Strongly Disagree (1), Disagree (2), Neither agree nor disagree (3), Agree (4), and Strongly agree (5).

  Up to 2 weeks

• Patient empathy, as measured by survey

  Likert-scale responses to empathy question: "I feel cared for during this interaction", on a scale from 1-5 with answer options of Strongly Disagree (1), Disagree (2), Neither agree nor disagree (3), Agree (4), and Strongly agree (5).

  Up to 2 weeks

Study Arms (6)

Arm A

OTHER

Each arm will receive 3 clinical scenarios spaced over time across 3 Sends. The 6 groups (Arms A-F) will be arranged with naming conventions as such: * First letter = A, B, or C where A = scenario 1, B = scenario 2, and C = scenario 3. * Second letter(s) = H or AI, where H = human response and AI = AI-written response to the patient question posed. * Third letter(s) = N, C, or H, where N = no disclosure, C = computer disclosure, and H = human disclosure. This refers to the disclosure at the bottom of the response message whereby the author is or is not disclosed. Arm A receives AHN in Send 1, BAIC in Send 2, and CHH in Send 3

Behavioral: Generative AI for electronic communication and disclosure

Arm B

OTHER

Each arm will receive 3 clinical scenarios spaced over time across 3 Sends. The 6 groups (Arms A-F) will be arranged with naming conventions as such: * First letter = A, B, or C where A = scenario 1, B = scenario 2, and C = scenario 3. * Second letter(s) = H or AI, where H = human response and AI = AI-written response to the patient question posed. * Third letter(s) = N, C, or H, where N = no disclosure, C = computer disclosure, and H = human disclosure. This refers to the disclosure at the bottom of the response message whereby the author is or is not disclosed. Arm B receives BHC in Send 1, CAIH in Send 2, and AAIN in Send 3

Behavioral: Generative AI for electronic communication and disclosure

Arm C

OTHER

Each arm will receive 3 clinical scenarios spaced over time across 3 Sends. The 6 groups (Arms A-F) will be arranged with naming conventions as such: * First letter = A, B, or C where A = scenario 1, B = scenario 2, and C = scenario 3. * Second letter(s) = H or AI, where H = human response and AI = AI-written response to the patient question posed. * Third letter(s) = N, C, or H, where N = no disclosure, C = computer disclosure, and H = human disclosure. This refers to the disclosure at the bottom of the response message whereby the author is or is not disclosed. Arm C receives CHC in Send 1, AHH in Send 2, and BAIN in Send 3

Behavioral: Generative AI for electronic communication and disclosure

Arm D

OTHER

Each arm will receive 3 clinical scenarios spaced over time across 3 Sends. The 6 groups (Arms A-F) will be arranged with naming conventions as such: * First letter = A, B, or C where A = scenario 1, B = scenario 2, and C = scenario 3. * Second letter(s) = H or AI, where H = human response and AI = AI-written response to the patient question posed. * Third letter(s) = N, C, or H, where N = no disclosure, C = computer disclosure, and H = human disclosure. This refers to the disclosure at the bottom of the response message whereby the author is or is not disclosed. Arm D receives AAIH in Send 1, BHN in Send 2, and CAIC in Send 3

Behavioral: Generative AI for electronic communication and disclosure

Arm E

OTHER

Each arm will receive 3 clinical scenarios spaced over time across 3 Sends. The 6 groups (Arms A-F) will be arranged with naming conventions as such: * First letter = A, B, or C where A = scenario 1, B = scenario 2, and C = scenario 3. * Second letter(s) = H or AI, where H = human response and AI = AI-written response to the patient question posed. * Third letter(s) = N, C, or H, where N = no disclosure, C = computer disclosure, and H = human disclosure. This refers to the disclosure at the bottom of the response message whereby the author is or is not disclosed. Arm E receives BAIH in Send 1, CHN in Send 2, and AHC in Send 3

Behavioral: Generative AI for electronic communication and disclosure

Arm F

OTHER

Each arm will receive 3 clinical scenarios spaced over time across 3 Sends. The 6 groups (Arms A-F) will be arranged with naming conventions as such: * First letter = A, B, or C where A = scenario 1, B = scenario 2, and C = scenario 3. * Second letter(s) = H or AI, where H = human response and AI = AI-written response to the patient question posed. * Third letter(s) = N, C, or H, where N = no disclosure, C = computer disclosure, and H = human disclosure. This refers to the disclosure at the bottom of the response message whereby the author is or is not disclosed. Arm F receives CAIN in Send 1, AAIC in Send 2, and BHH in Send 3

Behavioral: Generative AI for electronic communication and disclosure

Interventions

Generative AI for electronic communication and disclosure BEHAVIORAL

We will use a large language model such as GPT 3.5 to automatically generate responses to fictional messages to a physician. We will disclose whether the message was generated using this technology or not. There are 3 clinical scenarios and 6 pairs of human/AI response and human disclosure/AI disclosure/not disclosed that will test patient attitudes toward this technology.

Arm A; Arm B; Arm C; Arm D; Arm E; Arm F

Eligibility Criteria

• Age: 18 Years+
• Sex: all
• Healthy Volunteers: Yes
• Age Groups: Adult (18-64), Older Adult (65+)

You may qualify if:

• Member of the Duke Health Listens patient advocacy community

You may not qualify if:

• Age \< 18

Sponsors & Collaborators

• Duke University: lead

Study Sites (1)

Duke University Health System

Durham, North Carolina, 27710, United States

Location

Related Publications (1)

• Ayers JW, Poliak A, Dredze M, Leas EC, Zhu Z, Kelley JB, Faix DJ, Goodman AM, Longhurst CA, Hogarth M, Smith DM. Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media Forum. JAMA Intern Med. 2023 Jun 1;183(6):589-596. doi: 10.1001/jamainternmed.2023.1838.

  PMID: 37115527 BACKGROUND

MeSH Terms

Conditions

Communication

Condition Hierarchy (Ancestors)

Behavior

Study Officials

• Anand Chowdhury, MD, MMCi

  Duke University

  PRINCIPAL INVESTIGATOR

Study Design

• Study Type: interventional
• Phase: not applicable
• Allocation: RANDOMIZED
• Masking: SINGLE
• Who Masked: PARTICIPANT
• Masking Details: Participants will not be aware of the arm they are assigned to. There is no care provider or outcomes assessor in this study, as the patients will report their own perceptions in a survey.
• Purpose: OTHER
• Intervention Model: PARALLEL
• Sponsor Type: OTHER
• Responsible Party: SPONSOR

Model Details: * Participants will be randomly allocated to one of six arms. * Over the course of the study period, there will be multiple rounds of surveys, which will consist of a clinical scenario, a response (human or AI generated), and a disclosure statement (none, human, or AI). * All respondents within an arm will be assigned to a sequence of response-disclosure pairs prior to the first round of surveys.

Study Record Dates

• First Submitted: October 23, 2023
• First Posted: October 30, 2023
• Study Start: October 31, 2023
• Primary Completion: December 11, 2023
• Study Completion: December 11, 2023
• Last Updated: December 13, 2023

Record last verified: 2023-10

Data Sharing

• IPD Sharing: Will not share

Locations

US (1)