Developing and Evaluating a Machine-Learning Opioid Overdose Prediction & Risk-Stratification Tool in Primary Care

NCT06810076 | Active Not Recruiting

• 2 other identifiers: STUDY24040038R01DA050676
• Study Type: interventional
• Target: 674
• Locations: 1 country
• Sites: 1

Brief Summary

This clinical trial aims to evaluate the pilot implementation of a machine-learning (ML)-driven clinical decision support (CDS) tool designed to predict opioid overdose risk within the electronic health record (EHR) system at UF Health Internal Medicine and Family Medicine clinics in Gainesville, Florida. The study will use a pre- versus post-implementation design to compare outcomes within clinics, focusing on measures such as naloxone prescribing rates and opioid overdose occurrences. Researchers will also assess the usability, acceptability, and feasibility of the CDS tool through qualitative interviews with primary care clinicians (PCPs) in the participating clinics.

Conditions

Opiate Overdose; Opioid-Related Disorders; Mental Disorders; Narcotic-Related Disorders; Substance-related Disorders; Chemically-Induced Disorders

Keywords

Opiate overdose; Opioid-Related Disorders; Risk Evaluation and Mitigation; Machine Learning; Medical Order Entry Systems; Decision Support Systems, Clinical

Outcome Measures

Primary Outcomes (2)

• Composite patient-level outcomes related to opioids

  The CDS tool will generate an Overdose Prevention Alert (OPA) when a PCP signs an opioid order in Epic®. To evaluate the tool's effectiveness, researchers will conduct within-clinic comparisons (pre- vs. post-implementation) and examine a composite of patient-level outcomes post-implementation, including the proportion of patients having any of the following 6 outcomes: 1. receipt of a naloxone order or prescription fill; 2. absence of opioid overdose diagnoses and naloxone administration; 3. absence of ED visits or hospitalizations due to opioid overdose or OUD; 4. absence of overlapping opioid and benzodiazepine use; 5. absence of high-dose opioid use (average daily morphine milligram equivalent ≥50); 6. receipt of referrals to non-pharmacological pain management (e.g., physical therapy, chiropractic care).

  From enrollment and up to 12 months (3, 6, 12 months) post implementation of the OPA

• PCP's use feedback of the Overdose Prevention Alert (OPA)

  An online questionnaire for PCPs who interacted with OPA includes 12 Likert-scale items (4-point scale: 1 = Strongly Disagree to 4 = Strongly Agree) assessing OPA's acceptability, appropriateness, and feasibility: 1. OPA's information was clear. 2. OPA was easy to use. 3. OPA helps identify patients at increased overdose risk. 4. OPA helps understand patient's overdose risk. 5. OPA provides risk management recommendations. 6. OPA identifies the right patients with elevated overdose risk. 7. OPA notifies the correct healthcare team member (i.e., PCPs). 8. A pop-up alert is an appropriate notification approach. 9. Signing an opioid order is the right time for OPA. 10. Alert frequency is appropriate. 11. I prefer OPA over the legacy naloxone alert (see picture). 12. I want this OPA to continue to operate in my EHR. Mean scores (with standard deviations \[SD\]) will be calculated across all items, as well as individual average scores (SD).

  From enrollment and up to 7 months post implementation of the OPA

Secondary Outcomes (6)

• Receipt of a naloxone order or prescription fill

  From enrollment and up to 12 months (3, 6, 12 months) post implementation of the Overdose Prevention Alert (OPA)

• Absence of opioid overdose diagnoses and naloxone administration

  From enrollment and up to 12 months (3, 6, 12 months) post implementation

• Absence of ED visits or hospitalizations due to opioid overdose or OUD

  From enrollment and up to 12 months (3, 6, 12 months) post implementation

• Absence of overlapping opioid and benzodiazepine use

  From enrollment and up to 12 months (3, 6, 12 months) post implementation

• Absence of high-dose opioid use (average daily morphine milligram equivalent ≥50)

  From enrollment and up to 12 months (3, 6, 12 months) post implementation

Study Arms (1)

Overdose Prevention Alert (OPA) Intervention Arm

EXPERIMENTAL

The intervention arm will receive a ML CDS tool that provides interruptive alerts for patients at elevated risk of opioid overdose, triggered when a clinician signs an opioid order.

Behavioral: Machine Learning-Based Clinical Decision Support: Overdose Prevention Alert (OPA) Intervention

Interventions

Machine Learning-Based Clinical Decision Support: Overdose Prevention Alert (OPA) Intervention BEHAVIORAL

In this study, researchers will pilot test an interruptive, ML CDS tool for opioid overdose risk across thirteen primary care clinics at the UF Health in Gainesville, FL. When a patient is identified by the ML algorithm as having an elevated overdose risk and a PCP signs an opioid prescription for the patient, an Opioid Prevention Alert (OPA) will be triggered. The alert will include the rationale for the patient's elevated risk status and provide three risk mitigation recommendations: optimizing pain treatment and mental health support, reviewing and discussing risks with the patient, and offering naloxone annually if no prior naloxone order is found in the patient's record. PCPs can also select an override reason, such as the patient already has naloxone, declined the intervention, is not present/it is not the right time, or the alert is not relevant/other comments, when appropriate.

Overdose Prevention Alert (OPA) Intervention Arm

Eligibility Criteria

• Age: 18 Years+
• Sex: all
• Healthy Volunteers: No
• Age Groups: Adult (18-64), Older Adult (65+)

You may qualify if:

• For PCP level outcomes assessment
• PCPs
• practicing in any of the 13 participating clinics (10 UF Health Family Medicine clinics and 3 UF Health Internal Medicine) in Gainesville, Florida.
• For patient level outcomes assessment:
• are aged ≥18 years
• received any opioid prescription in the past year prior to their clinic visit.

You may not qualify if:

• had malignant cancer diagnosis or hospice care prior to study enrollment

Sponsors & Collaborators

• University of Pittsburgh: lead
• National Institute on Drug Abuse (NIDA): collaborator
• Applied Decision Science: collaborator

Study Sites (1)

University of Florida Health Internal Medicine and Family Medicine

Gainesville, Florida, 32608, United States

Location

Related Publications (6)

• Hong JJ,Wilson DL,Nguyen K,Gellad WF,Diiulio J,Militello L,Yan S,Harle CA,Nelson D,Rosenberg EI,Schmidt S,Chang CH,Cochran G,Wu Y,Staras SAS,Kuza C,Lo-Ciganic WH

  BACKGROUND

• Lo-Ciganic WH, Donohue JM, Yang Q, Huang JL, Chang CY, Weiss JC, Guo J, Zhang HH, Cochran G, Gordon AJ, Malone DC, Kwoh CK, Wilson DL, Kuza CC, Gellad WF. Developing and validating a machine-learning algorithm to predict opioid overdose in Medicaid beneficiaries in two US states: a prognostic modelling study. Lancet Digit Health. 2022 Jun;4(6):e455-e465. doi: 10.1016/S2589-7500(22)00062-0.

  PMID: 35623798 BACKGROUND

• Lo-Ciganic WH, Huang JL, Zhang HH, Weiss JC, Kwoh CK, Donohue JM, Gordon AJ, Cochran G, Malone DC, Kuza CC, Gellad WF. Using machine learning to predict risk of incident opioid use disorder among fee-for-service Medicare beneficiaries: A prognostic study. PLoS One. 2020 Jul 17;15(7):e0235981. doi: 10.1371/journal.pone.0235981. eCollection 2020.

  PMID: 32678860 BACKGROUND

• Lo-Ciganic WH, Huang JL, Zhang HH, Weiss JC, Wu Y, Kwoh CK, Donohue JM, Cochran G, Gordon AJ, Malone DC, Kuza CC, Gellad WF. Evaluation of Machine-Learning Algorithms for Predicting Opioid Overdose Risk Among Medicare Beneficiaries With Opioid Prescriptions. JAMA Netw Open. 2019 Mar 1;2(3):e190968. doi: 10.1001/jamanetworkopen.2019.0968.

  PMID: 30901048 BACKGROUND

• Militello LG, Diiulio J, Wilson DL, Nguyen KA, Harle CA, Gellad W, Lo-Ciganic WH. Using human factors methods to mitigate bias in artificial intelligence-based clinical decision support. J Am Med Inform Assoc. 2025 Feb 1;32(2):398-403. doi: 10.1093/jamia/ocae291.

  PMID: 39569464 BACKGROUND

• Nguyen K, Wilson DL, Diiulio J, Hall B, Militello L, Gellad WF, Harle CA, Lewis M, Schmidt S, Rosenberg EI, Nelson D, He X, Wu Y, Bian J, Staras SAS, Gordon AJ, Cochran J, Kuza C, Yang S, Lo-Ciganic W. Design and development of a machine-learning-driven opioid overdose risk prediction tool integrated in electronic health records in primary care settings. Bioelectron Med. 2024 Oct 18;10(1):24. doi: 10.1186/s42234-024-00156-3.

  PMID: 39420438 BACKGROUND

Related Links

• Center for Pharmaceutical Policy \& Prescribing website

MeSH Terms

Conditions

Opiate Overdose; Opioid-Related Disorders; Narcotic-Related Disorders; Substance-Related Disorders; Chemically-Induced Disorders; Mental Disorders

Interventions

Methods

Condition Hierarchy (Ancestors)

Drug Overdose; Prescription Drug Misuse; Drug Misuse

Intervention Hierarchy (Ancestors)

Investigative Techniques

Study Officials

• Wei-Hsuan Lo-Ciganic, PhD

  Division of General Internal Medicine, School of Medicine, University of Pittsburgh, Pittsburgh, PA

  PRINCIPAL INVESTIGATOR

Study Design

• Study Type: interventional
• Phase: not applicable
• Allocation: NA
• Masking: NONE
• Purpose: HEALTH SERVICES RESEARCH
• Intervention Model: SINGLE GROUP
• Sponsor Type: OTHER
• Responsible Party: PRINCIPAL INVESTIGATOR
• PI Title: Professor

Model Details: This single-arm clinical trial employs a pre- and post-implementation pilot evaluation design to assess the usability, acceptability, and feasibility of implementing a ML-driven overdose CDS tool across thirteen UF Health primary care clinics (3 internal medicine and 6 family medicine clinics in Gainesville, Florida). The CDS tool will generate an Overdose Prevention Alert (OPA) when a PCP signs an opioid order in Epic® for patients at elevated risk of opioid overdose identified by ML algorithm.

Study Record Dates

• First Submitted: January 27, 2025
• First Posted: February 5, 2025
• Study Start: April 8, 2025
• Primary Completion (Estimated): October 7, 2026
• Study Completion (Estimated): October 7, 2026
• Last Updated: April 13, 2026

Record last verified: 2026-04

Data Sharing

• IPD Sharing: Will not share

Locations

US (1)