Working to Implement and Sustain Digital Outcome Measures

NCT04096274 | Completed Results DMC

• 2 other identifiers: 8424R01MH119127
• Study Type: interventional
• Target: 686
• Locations: 1 country
• Sites: 4

Brief Summary

This study will investigate the effects of an organizational implementation strategy called Leadership and Organizational Change for Implementation (LOCI), relative to training and technical assistance only, on fidelity to, and youth service outcomes of, a well-established digital measurement-based care intervention called the Outcomes Questionnaire-Analyst in outpatient community mental health clinics.

Conditions

Implementation; Behavioral Symptoms

Keywords

LOCI; measurement-based care; evidence-based practice; behavioral health; youth mental health; leadership; implementation science

Outcome Measures

Primary Outcomes (3)

• Percentage Fidelity to the OQ-A System Experienced by the Youth (0-100%)

  Fidelity to the OQ-A will be measured by using electronic meta-data from the OQ-A system combined with caregiver reported information on the number of sessions the youth attended. For each youth, a fidelity index will be generated that represents the product of two quantities: (a) the youth's completion rate (i.e., number of measures administered relative to the number of sessions attended within the 6-month observation period), and (b) the youth's viewing rate (i.e., the number of feedback reports viewed by the clinician relative to the number of measures administered). Note that this product is equivalent to the ratio of viewed feedback reports to total sessions; it represents an events/trials proportion. MBC fidelity index scores summarize the level of MBC fidelity experienced by each youth (range=0-1). Higher scores indicate the youth experienced greater fidelity to MBC.

  0-6 months after youth's baseline/ entry into treatment

• Change From Baseline to 6-months in Youth Total Problems Score on the Short Form Assessment for Children (SAC) - Phase I Cohort

  The SAC Total Problem Score is a 48-item measure of youth internalizing (e.g., anxious, depressed) and externalizing (e.g., aggressive, noncompliant, overactive) behaviors (range=0-96) completed by caregivers of youth. Total Problem Score was assessed at baseline (youth's entry into treatment) and monthly for 6 months, change from baseline to month 6 is reported.

  0-6 months after youth's baseline/ entry into treatment

• Change From Baseline to 6-months in Youth Total Problems Score on the Short-form Assessment for Children (SAC) - Phase II Cohort

  The SAC Total Problem Score is a 48-item measure of youth internalizing (e.g., anxious, depressed) and externalizing (e.g., aggressive, noncompliant, overactive) behaviors (range=0-96) completed by caregivers of youth. Total Problem Score was assessed at baseline (youth's entry into treatment) and monthly for 6 months, change from baseline to month 6 is reported.

  0-6 months after youth's baseline/ entry into treatment

Study Arms (2)

LOCI (Intervention)

EXPERIMENTAL

In clinics assigned to the LOCI condition, executives and first-level leaders will receive leadership training and coaching to support implementation of the OQ-A system. In addition, leaders and clinicians in this condition will receive training and technical assistance to implement the OQ-A measurement-based care system.

Behavioral: Leadership for Organization Change and Implementation (LOCI); Behavioral: Training and Technical Assistance Only

Training and Technical Assistance only (Control)

ACTIVE COMPARATOR

In clinics assigned to the control group, leaders and clinicians will receive training and technical assistance to implement the OQ-A measurement-based care system. In addition, to support enrollment in this condition, leaders in this condition will be offered access to general web-based leadership seminars.

Behavioral: Training and Technical Assistance Only

Interventions

Leadership for Organization Change and Implementation (LOCI) BEHAVIORAL

LOCI is a multicomponent implementation strategy that engages organizational executives and first-level leaders (i.e., those who administratively supervise clinicians) to build an organizational climate to support the implementation of a focal evidence-based practice (EBP) with fidelity. In this study, the focal EBP is the OQ-A system. LOCI includes two overarching components: (1) monthly organizational strategy meetings between executives and LOCI consultants/trainers to develop and embed policies, procedures, and practices that support implementation of a focal EBP, and (2) training and coaching for first-level leaders, to develop their skills in leading implementation. The aim of these components is to develop an organizational implementation climate in which clinicians' perceive that use of the OQ-A with high fidelity is expected, supported, and rewarded.

LOCI (Intervention)

Training and Technical Assistance Only BEHAVIORAL

All leaders and clinicians in participating clinics will receive standardized OQ-A training and technical assistance provided by the OQ-A purveyor organization. This includes an initial, 6-hr, in-person OQ-A training; two, live, virtual, 1-hr booster trainings, offered 3 and 5 months after the initial training; and, year-round technical assistance from the OQ-A purveyor organization. Technical assistance includes virtual training sessions, online library of training videos, and customer care representative for technical support. In addition, to encourage participation in the study, a set of four 1-hr, web-based general leadership seminars will be offered to leaders in the control condition. These will cover topics ranging from effective leadership, to giving effective feedback.

LOCI (Intervention); Training and Technical Assistance only (Control)

Eligibility Criteria

• Age: 4 Years+
• Sex: all
• Healthy Volunteers: Yes
• Age Groups: Child (0-17), Adult (18-64), Older Adult (65+)

You may qualify if:

• Provide outpatient psychotherapy services to children ages 4 to 18 who have emotional and/ or behavioral disorders and their families
• Have at least 3 full time equivalent clinicians on staff
• Not currently implementing a digital measurement-based care system clinic wide
• \. Identified as Chief Executive Officer, Executive Director, or high-level administrator at an enrolled clinic
• \. Identified as a clinical supervisor or clinical work-group supervisor/ leader at an enrolled clinic
• Employed as staff at participating clinic
• Provides outpatient mental health services (psychotherapy) to youth clients
• Child is ages 4 to 18 years at intake,
• Child has been diagnosed with an emotional or behavioral disorder by clinic staff
• Clinical staff at the site determined that the youth's treatment needs can be appropriately addressed by the clinic

Sponsors & Collaborators

• Boise State University: lead
• University of California, San Diego: collaborator
• University of Central Florida: collaborator
• University of Pennsylvania: collaborator
• National Institute of Mental Health (NIMH): collaborator

Study Sites (4)

University of California, San Diego

San Diego, California, 92093, United States

Location

University of Central Florida

Orlando, Florida, 32816, United States

Location

Boise State University

Boise, Idaho, 83725, United States

Location

University of Pennsylvania

Philadelphia, Pennsylvania, 19104, United States

Location

Related Publications (3)

• Williams NJ, Ehrhart MG, Aarons GA, Esp S, Sklar M, Carandang K, Vega NR, Brookman-Frazee L, Marcus SC. Improving measurement-based care implementation in youth mental health through organizational leadership and climate: a mechanistic analysis within a randomized trial. Implement Sci. 2024 Mar 28;19(1):29. doi: 10.1186/s13012-024-01356-w.

  PMID: 38549122 DERIVED

• Williams NJ, Marcus SC, Ehrhart MG, Sklar M, Esp SM, Carandang K, Vega N, Gomes AE, Brookman-Frazee L, Aarons GA. Randomized Trial of an Organizational Implementation Strategy to Improve Measurement-Based Care Fidelity and Youth Outcomes in Community Mental Health. J Am Acad Child Adolesc Psychiatry. 2024 Oct;63(10):991-1004. doi: 10.1016/j.jaac.2023.11.010. Epub 2023 Dec 7.

  PMID: 38070868 DERIVED

• Choy-Brown M, Williams NJ, Ramirez N, Esp S. Psychometric evaluation of a pragmatic measure of clinical supervision as an implementation strategy. Implement Sci Commun. 2023 Apr 6;4(1):39. doi: 10.1186/s43058-023-00419-1.

  PMID: 37024945 DERIVED

Related Links

• Information about Leadership for Organizational Change for Implementation (LOCI).
• Information about the digital measurement-based care system, OQ-A.

MeSH Terms

Conditions

Behavioral Symptoms

Interventions

Leadership

Condition Hierarchy (Ancestors)

Behavior

Intervention Hierarchy (Ancestors)

Personnel Management; Organization and Administration; Health Services Administration

Results Point of Contact

• Title: Dr. Nathaniel Williams
• Organization: Boise State University

natewilliams@boisestate.edu

2084263145

Study Officials

• Nate Williams, PhD

  Boise State University

  PRINCIPAL INVESTIGATOR

Publication Agreements

• PI is Sponsor Employee: No
• Restrictive Agreement: No

Study Design

• Study Type: interventional
• Phase: not applicable
• Allocation: RANDOMIZED
• Masking: DOUBLE
• Who Masked: PARTICIPANT, CARE PROVIDER
• Masking Details: Clinicians and caregivers of youth were naïve to condition; however, masking of clinic leaders was not possible due to the nature of the LOCI strategy (which entails leadership training and consultation).
• Purpose: HEALTH SERVICES RESEARCH
• Intervention Model: PARALLEL
• Sponsor Type: OTHER
• Responsible Party: SPONSOR

Study Record Dates

• First Submitted: September 9, 2019
• First Posted: September 19, 2019
• Study Start: October 1, 2019
• Primary Completion: September 17, 2022
• Study Completion: September 17, 2022
• Last Updated: February 20, 2024
• Results First Posted: February 20, 2024

Record last verified: 2022-07

Data Sharing

• IPD Sharing: Will not share

Locations

US (4)