K-ORCA: Testing a Decision Support Tool and Group Process for Selecting Interventions

NCT06375551 | Recruiting DMC

• 2 other identifiers: 1171-0623K01MH128761
• Study Type: interventional
• Target: 80
• Locations: 1 country
• Sites: 1

Brief Summary

This proposal responds to NIMH Objective 4.2.c to develop "decision-support tools and technologies that increase the effectiveness and continuous improvement of mental health interventions" by leveraging the Family First Prevention Services Act (FFPSA) policy opportunity. First, a web-based platform to host (a) a decision-support tool and (b) automated facilitation for group decisions with the tool will be developed with state partners' feedback. Next, decision makers leading their states' FFPSA quality improvement efforts will be engaged to pilot a decision-support intervention comprised of the tool and live or automated facilitation, and to evaluate the implementation quality of evidence-based programs adopted with the decision-support intervention.

Conditions

Child Abuse; Decision Making; Child Welfare; Decision Making, Shared; Decision Support Technique; Social Facilitation; Implementation Science; Policy; Organizations; Consensus; Mental Health; Family

Outcome Measures

Primary Outcomes (11)

• Mean changes in decision assumptions, decision goals Pre and Immediately Post-intervention

  A two-item, quantitative measure will inquire about assumptions and goals in the decision to adopt an intervention. This uses a 5-point Likert scale ranging from 1 (Strongly disagree) to 5 (Strongly Agree). This outcome is part of Aim 2b: Decision quality based on intervention fit to local context.

  Baseline; Immediately Post-intervention

• Mean changes in decision experience quality pre-post intervention

  This will be measured across five validated, quantitative sub-scales from a measure of community partnered research: Partner Values, Synergy, Conflict and Cooperation, Participation, Participatory decision-making. Each scale has a range of 1 (Strongly Disagree) to 5 (Strongly Agree).

  Baseline; Immediately Post-intervention

• Mean changes in decision commitment pre-post intervention

  A two-item, Likert-scale measure regarding the decision maker's confidence in and commitment to implementing the selected intervention(s) will be gathered using a two-item measure. This uses a 5-point Likert scale ranging from 1 (Strongly disagree) to 5 (Strongly Agree). These scales relate to Aim 2a: decision experience quality.

  Baseline; Immediately Post-intervention

• Mean changes in perceived potential community health impact pre-post intervention

  This will be measured across a validated, quantitative sub-scale from a measure for community partnered research: Community Health Improvement. This scale has a range of 1 (Strongly Disagree) to 5 (Strongly Agree). This is part of Aim 2b: Decision quality.

  Baseline; Immediately Post-intervention; 12-month follow-up

• Descriptive changes in ordinal intervention rankings assisted (with ORCA) compared to unassisted (no ORCA)

  Decision makers will rank interventions in an ordinal manner at baseline using a study-specific survey. Rankings will be calculated post-intervention by ORCA as a function of completing the ORCA tool. The number of ordinal rankings will depend on the number of interventions considered, to be determined by participants. A rank of 1 = highest, followed by 2 ...n. This is part of Aim 2c: Decision quality.

  Baseline; Immediately Post-intervention

• Qualitatively described rationale for program adoption from pre-post intervention

  This is a study-specific, short, open-ended survey to understand decision makers' rationale for adopting programs during initial Family First Prevention Services Act decisions. There is no scale for the items, given the qualitative, open-ended nature. These are one survey of similar, but related constructs of intervention feasibility, acceptability, and appropriateness. This is part of Aim 2c: Decision quality.

  Baseline; Immediately Post-intervention

• Descriptive, mean group-level changes in perceived feasibility of intervention Pre-post intervention

  The validated, quantitative Feasibility of Intervention Measure (FIM) will be used to assess perceived implementation feasibility for each intervention considered with ORCA. Scale ranges from 1 (Completely Disagree) to 5 (Completely Agree). This outcome is part of Aim 2b regarding decision quality operationalized as intervention fit to local context.

  Baseline; Immediately Post-intervention; 12-month post-intervention

• Descriptive, mean group-level changes in perceived acceptability of intervention for implementation

  The validated, quantitative Acceptability of Intervention Measure (AIM) will be used to assess perceived implementation feasibility for each intervention considered with ORCA. Scale ranges from 1 (Completely Disagree) to 5 (Completely Agree). This outcome is part of Aim 2b regarding decision quality operationalized as intervention fit to local context.

  Baseline; Immediately Post-intervention; 12-month post-intervention

• Descriptive, mean group-level changes in perceived appropriateness of intervention for implementation

  The validated, quantitative Intervention Appropriateness Measure (IAM) will be used to assess perceived acceptability for each intervention considered with ORCA. Scale ranges from 1 (Completely Disagree) to 5 (Completely Agree). This outcome is part of Aim 2b regarding decision quality operationalized as intervention fit to local context.

  Baseline; Immediately Post-intervention; 12-month post-intervention

• Descriptive differences in implementation process fidelity (activities completed) for interventions adopted with ORCA

  A validated measure of implementation activities across 8 stages and 3 phases will be used (Universal Stages of Implementation Completion). Dates are recorded for each activity to be reported or as missing the date, but still being completed. One of the results for the calculated scores is the proportion of activities completed. A higher proportion of activities is generally positive.

  Immediately Post-intervention; bi-weekly through 12-month post-intervention

• Descriptive differences in implementation process fidelity (timing) for interventions adopted with ORCA

  A validated measure of implementation activities across 8 stages and 3 phases will be used (Universal Stages of Implementation Completion). Dates are recorded for each activity to be reported or as missing the date, but still being completed. One of the results for the calculated scores is the duration (time elapsed). The ideal duration depends on the proportion score.

  Immediately Post-intervention; bi-weekly through 12-month post-intervention

Secondary Outcomes (4)

• Qualitative changes in decision reasoning pre- post-intervention

  Baseline; Immediately Post-intervention

• Qualitative changes in decision goals pre- post-intervention

  Baseline; Immediately Post-intervention

• Qualitative changes in decision consensus pre- post-intervention

  Baseline; Immediately Post-intervention

• Time to adoption or re-adoption of interventions with ORCA

  Baseline to adoption of an intervention for implementation, assessed for up to 24 months

Study Arms (2)

ORCA-A

ACTIVE COMPARATOR

Decision Makers receiving automated facilitation to accompany ORCA

Behavioral: Optimizing Responses with Collaborative Assessments-Automated (ORCA-A)

ORCA-L

ACTIVE COMPARATOR

Decision Makers receiving live facilitation to accompany ORCA

Behavioral: Optimizing Responses with Collaborative Assessments-Live (ORCA-L)

Interventions

Optimizing Responses with Collaborative Assessments-Automated (ORCA-A) BEHAVIORAL

Participants will receive automated facilitation in the ORCA platform. This will entail prompts for group discussion based on group model building scripts and decision hygiene approaches proposed by Kahneman, Sibony, and Sunstein (2021). Participants will have the option of using these prompts to discuss ORCA results in group decision discussions. Optimizing Responses with Collaborative Assessments (ORCA) is a technical decision support tool based in multi-criteria decision analysis. Decision makers rate and compare evidence-based programs or other practices with ORCA, then prioritize practices for local implementation based on ORCA results. ORCA will be on a virtual platform to allow for web-based completion, automated analysis, and automated facilitation.

ORCA-A

Optimizing Responses with Collaborative Assessments-Live (ORCA-L) BEHAVIORAL

A facilitator will guide group decision discussions using group model building scripts and decision hygiene approaches proposed by Kahneman, Sibony, and Sunstein (2021). Facilitation will be either in-person or virtual, but occur "live" as in during real-time. Optimizing Responses with Collaborative Assessments (ORCA) is a technical decision support tool based in multi-criteria decision analysis. Decision makers rate and compare evidence-based programs or other practices with ORCA, then prioritize practices for local implementation based on ORCA results. ORCA will be on a virtual platform to allow for web-based completion, automated analysis, and automated facilitation.

ORCA-L

Eligibility Criteria

• Age: 18 Years+
• Sex: all
• Healthy Volunteers: Yes
• Age Groups: Adult (18-64), Older Adult (65+)

You may qualify if:

• Non-institutionalized
• Adults (18 years or older).
• English-Speaking
• Aim 1:
• Involved in initial state decision-making related to Family First Prevention Services Act.
• Willing to participate in two data collection occasions.
• Aim 2:
• Non-institutionalized
• Adults (18 years or older). English-Speaking.
• Involved in ongoing state decision-making related to Family First Prevention Services Act.
• Willing to participate in three measurement occasions.
• Aim 3:
• Willing to participate in multiple (bi-weekly up to two years) measurement occasions.

You may not qualify if:

• Not involved in or potentially influencing child welfare intervention decisions

Sponsors & Collaborators

• Chestnut Health Systems: lead
• National Institute of Mental Health (NIMH): collaborator

Study Sites (1)

Chestnut Health Systems

Eugene, Oregon, 97401, United States

RECRUITING

Related Publications (4)

• Alley ZM, Chapman JE, Schaper H, Saldana L. The relative value of Pre-Implementation stages for successful implementation of evidence-informed programs. Implement Sci. 2023 Jul 21;18(1):30. doi: 10.1186/s13012-023-01285-0.

  PMID: 37480144 BACKGROUND

• Oetzel JG, Wallerstein N, Duran B, Sanchez-Youngman S, Nguyen T, Woo K, Wang J, Schulz A, Keawe'aimoku Kaholokula J, Israel B, Alegria M. Impact of Participatory Health Research: A Test of the Community-Based Participatory Research Conceptual Model. Biomed Res Int. 2018 Apr 24;2018:7281405. doi: 10.1155/2018/7281405. eCollection 2018.

  PMID: 29854784 BACKGROUND

• Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, Boynton MH, Halko H. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci. 2017 Aug 29;12(1):108. doi: 10.1186/s13012-017-0635-3.

  PMID: 28851459 BACKGROUND

• Cruden G, Frerichs L, Powell BJ, Lanier P, Brown CH, Lich KH. Developing a Multi-Criteria Decision Analysis Tool to Support the Adoption of Evidence-Based Child Maltreatment Prevention Programs. Prev Sci. 2020 Nov;21(8):1059-1064. doi: 10.1007/s11121-020-01174-8. Epub 2020 Oct 11.

  PMID: 33040271 BACKGROUND

MeSH Terms

Conditions

Social Facilitation; Psychological Well-Being

Condition Hierarchy (Ancestors)

Social Behavior; Behavior; Personal Satisfaction

Study Officials

• Gracelyn Cruden, PhD

  Chestnut Health Systems

  PRINCIPAL INVESTIGATOR

Central Study Contacts

Gracelyn Cruden, PhD

CONTACT

843-513-9928; gcruden@chestnut.org

Kelli Wright

CONTACT

kwright@chestnut.org

Study Design

• Study Type: interventional
• Phase: not applicable
• Allocation: RANDOMIZED
• Masking: NONE
• Purpose: HEALTH SERVICES RESEARCH
• Intervention Model: PARALLEL
• Sponsor Type: OTHER
• Responsible Party: PRINCIPAL INVESTIGATOR
• PI Title: Research Scientist

Study Record Dates

• First Submitted: November 20, 2023
• First Posted: April 19, 2024
• Study Start: May 31, 2026
• Primary Completion (Estimated): July 31, 2027
• Study Completion (Estimated): July 31, 2027
• Last Updated: May 18, 2026

Record last verified: 2026-05

Data Sharing

• IPD Sharing: Will not share

Locations

US (1)