Using Surveys to Examine the Association of Exposure to ML Mortality Risk Predictions With Medical Oncologists' Prognostic Accuracy and Decision-making

NCT06463977 | Completed

• 2 other identifiers: UPCC 10524850382
• Study Type: observational
• Target: 52
• Locations: 1 country
• Sites: 1

Brief Summary

Nearly half of cancer patients in the US will receive care that is inconsistent with their wishes prior to death. Early advanced care planning (ACP) and palliative care improve goal-concordant care and symptoms and reduce unnecessary utilization. A promising strategy to increase ACP and palliative care is to identify patients at risk of mortality earlier in the disease course in order to target these services. Machine learning (ML) algorithms have been used in various industries, including medicine, to accurately predict risk of adverse outcomes and direct earlier resources. "Human-machine collaborations" - systems that leverage both ML and human intuition - have been shown to improve predictions and decision-making in various situations, but it is not known whether human-machine collaborations can improve prognostic accuracy and lead to greater and earlier ACP and palliative care. In this study, we contacted a national sample of medical oncologists and invited them complete a vignette-based survey. Our goal was to examine the association of exposure to ML mortality risk predictions with clinicians' prognostic accuracy and decision-making. We presented a series of six vignettes describing three clinical scenarios specific to a patient with advanced non-small cell lung cancer (aNSCLC) that differ by age, gender, performance status, smoking history, extent of disease, symptoms and molecular status. We will use these vignette-based surveys to examine the association of exposure to ML mortality risk predictions with medical oncologists' prognostic accuracy and decision-making.

Conditions

Oncology

Outcome Measures

Primary Outcomes (1)

• Prognostic accuracy as assessed via survey

  Prognostic estimates were measured using two items administered after Parts 1 and 2 of each of the 3 vignettes: 1. What is your anticipated life expectancy for this patient, in months? 2. What do you think is the likelihood that she will die within 12 months? Please provide a percentage on a scale of 0% to 100%. Accurate prognoses were defined as whether the reported life expectancy estimate was within 33% of the LCPI estimate, as modified after the focus groups. Participants answered the first question in months and the second question as a percentage between 0-100%.

  Up to 3 months

Secondary Outcomes (2)

• Advance care planning decisions as assessed via survey

  Up to 3 months

• Palliative care referral as assessed via survey

  Up to 3 months

Study Arms (6)

1A 2B 3C

1\. Intermediate; 1.A. Reference dependent; 2. Poor; 2.B. Absolute prognosis; 3. Good; 3.C. Both

Other: Survey

1A 2C 3B

1\. Intermediate; 1.A. Reference dependent; 2. Poor; 2.C. Both; 3. Good; 3.B. Absolute prognosis

Other: Survey

1B 2A 3C

1\. Intermediate; 1.B. Absolute; 2. Poor; 2.A. Reference dependent; 3. Good; 3.C. Both

Other: Survey

1B 2C 3A

1\. Intermediate; 1.B. Absolute; 2. Poor; 2.C. Both; 3. Good; 3.A. Reference dependent

Other: Survey

1C 2A 3B

1\. Intermediate; 1.C. Both; 2. Poor; 2.A. Reference dependent; 3. Good; 3.B. Absolute

Other: Survey

1C 2B 3A

1\. Intermediate; 1.C. Both; 2. Poor; 2.B. Absolute; 3. Good; 3.A. Reference dependent

Other: Survey

Interventions

Survey OTHER

The study consisted of a 3 × 3 online factorial experiment employing a survey instrument hosted via Qualtrics presenting describing three patient vignettes. The three patient vignettes varied by various clinical characteristics including age, gender, performance status, smoking history, extent of disease, symptoms and molecular status. Each patient had advanced non-small cell lung cancer (aNSCLC). Each vignette had two parts: Part 1 described the case history for one of the three patients, after which prognostic estimates and medical decision-making was assessed (i.e. 1, 2, 3). Part 2 immediately followed and described the same vignette from the same patient with added information from a hypothetical ML predictive algorithm (i.e. A, B, C). The order of the vignettes in each survey was randomized with regard to presentation strategies for the ML risk predictions, so that there were 6 versions of the survey to which each participant was randomized.

1A 2B 3C; 1A 2C 3B; 1B 2A 3C; 1B 2C 3A; 1C 2A 3B; 1C 2B 3A

Eligibility Criteria

• Sex: all
• Healthy Volunteers: No
• Age Groups: Child (0-17), Adult (18-64), Older Adult (65+)
• Sampling Method: Non-Probability Sample

Study Population

The study population consisted of a convenience sample of practicing medical oncologists who treated lung cancer in the US. We recruited medical oncologists through direct emails to the Principal Investigator's personal contacts (n=29); direct messages via Doximity, an online networking service for medical professionals (n=17), using a "thoracic oncology" ; and direct messages via X (formerly Twitter) (n=4). Efforts were taken to sample equally from 4 US geographic regions (Northeast, South, Midwest, West).

You may qualify if:

• Medical oncologists who treat lung cancer

You may not qualify if:

• Medical oncologists who do not see lung cancer patients

Sponsors & Collaborators

• Abramson Cancer Center at Penn Medicine: lead

Study Sites (1)

Abramson Cancer Center of the University of Pennsylvania

Philadelphia, Pennsylvania, 19104, United States

Location

MeSH Terms

Conditions

Neoplasms

Interventions

Surveys and Questionnaires

Intervention Hierarchy (Ancestors)

Data Collection; Epidemiologic Methods; Investigative Techniques; Health Care Evaluation Mechanisms; Quality of Health Care; Health Care Quality, Access, and Evaluation; Public Health; Environment and Public Health

Study Design

• Study Type: observational
• Observational Model: OTHER
• Time Perspective: CROSS SECTIONAL
• Sponsor Type: OTHER
• Responsible Party: SPONSOR

Study Record Dates

• First Submitted: April 30, 2024
• First Posted: June 18, 2024
• Study Start: March 13, 2023
• Primary Completion: July 31, 2023
• Study Completion: December 31, 2023
• Last Updated: November 21, 2024

Record last verified: 2024-11

Data Sharing

• IPD Sharing: Will not share

Locations

US (1)