Local Participatory Systems Dynamics to Increase Reach of Evidence Based Addiction and Mental Health Care

NCT04356274 | Enrolling By Invitation DMC

• 1 other identifier: ZIM_0002
• Study Type: interventional
• Target: 720
• Locations: 1 country
• Sites: 1

Brief Summary

The most common reasons Veterans seek VA addiction and mental health care is for help with opioid and alcohol misuse, depression and PTSD. Research evidence has established highly effective treatments that prevent relapse, overdose and suicide, but even with policy mandates, performance metrics, and electronic health records to fix the problem, these treatments may only reach 3-28% of patients. This study tests participatory business engineering methods (Participatory System Dynamics) that engage patients, providers and policy makers against the status quo approaches, such as data review, and will determine if participatory system dynamics works, why it works, and whether it can be applied in many health care settings to guarantee patient access to the highest quality care and better meet the addiction and mental health needs of Veterans and the U.S. population.

Conditions

PTSD; Depression; Alcohol Use Disorder; Opioid Use Disorder

Keywords

implementation science; quality improvement; participatory system dynamics; audit and feedback; evidence-based psychotherapy; evidence-based pharmacotherapy; addiction; mental health/behavioral health

Outcome Measures

Primary Outcomes (3)

• Proportion of patients diagnosed with alcohol use disorder, depression, opioid use disorder, or PTSD who meet evidence-based psychotherapy and pharmacotherapy initiation and course measures divided by total number of patients with these diagnoses

  Initiation of an evidence-based practice is indicated by an evidence-based psychotherapy template or evidence-based pharmacotherapy prescription after intake. Adequate course is based on receiving an adequate number of evidence-based psychotherapy sessions to be a "completer" (typically 8 sessions) or enough refills for a guideline-recommended adequate trial of each medication (varies by medication). Data is gathered based on electronic health record data from the VA Corporate Data Warehouse (CDW).

  Pre-/Post- 12-month period average of evidence-based practice reach (24 months total observation)

• Proportion of completed evidence-based practice templates during sessions with a relevant CPT code

  We will study 5 evidence-based psychotherapies: 3 for depression (Cognitive Behavior Therapy (CBT-D), Acceptance and Commitment Therapy (ACT), and Interpersonal Psychotherapy (IPT)) and 2 for PTSD (Prolonged Exposure (PE) and Cognitive Processing Therapy (CPT)). Data is gathered based on electronic health record data from the Corporate Data Warehouse (CDW).

  Pre-/Post- 12-month period average of evidence-based practice reach (24 months total observation)

• Proportion of combination of prescriptions placed with the VA pharmacy and sessions with a relevant CPT code

  We will study 8 evidence-based pharmacotherapies: 2 for depression (84 and 180 days therapeutic continuity at new antidepressant start), 2 for Opioid Use Disorder (OUD) (methadone and buprenorphine), and 4 for Alcohol Use Disorder (AUD) (Acamprosate, Disulfiram, Naltrexone, and Topiramate). Data is gathered based on electronic health record data from the Corporate Data Warehouse (CDW).

  Pre-/Post- 12-month period average of evidence-based practice reach (24 months total observation)

Secondary Outcomes (10)

• Degree of acceptability of intervention assessed by the Acceptability of Intervention Measure (AIM) [followed by its scale information in the Description]

  At 6 months

• Degree of appropriateness of intervention assessed by the Intervention Appropriateness Measure (IAM) [followed by its scale information in the Description]

  At 6 months

• Degree of feasibility of intervention assessed by the Feasibility of Intervention Measure (FIM) [followed by its scale information in the Description]

  At 6 months

• Patient Aligned Care team Burnout Measure (PACT) [followed by its scale information in the Description]

  At baseline and 6 months

• Learning Organization Survey (LOS-27) [followed by its scale information in the Description]

  At baseline and 6 months

Study Arms (2)

Participatory System Dynamics (PSD)

EXPERIMENTAL

12 clinics assigned to PSD

Other: Participatory System Dynamics (PSD)

Audit and Feedback (AF)

EXPERIMENTAL

12 clinics assigned to AF

Other: Audit and Feedback (AF)

Interventions

Participatory System Dynamics (PSD) OTHER

Participatory system dynamics is a facilitated health care quality improvement or evidence-based practice implementation strategy that includes frontline addiction and mental health staff running simulations of clinic improvement strategies to find the best approaches for improving the reach of evidence-based psychotherapy and evidence-based pharmacotherapy.

Participatory System Dynamics (PSD)

Audit and Feedback (AF) OTHER

Audit and feedback is a health care quality improvement or evidence-based practice implementation strategy that includes frontline addiction and mental health staff reviewing clinical care team data to find the best approaches for improving the reach of evidence-based psychotherapy and evidence-based pharmacotherapy.

Audit and Feedback (AF)

Eligibility Criteria

• Age: 18 Years+
• Sex: all
• Healthy Volunteers: Yes
• Age Groups: Adult (18-64), Older Adult (65+)

You may qualify if:

• VA divisions and community-based outpatient clinics (CBOCs) or 'clinics' from regional VA health systems
• Must be below the overall VA quality median (as assessed by the Strategic Analytics for Improvement and Learning or SAIL), which includes 3 of 8 SAIL measures associated with four evidence-based psychotherapies and three evidence-based pharmacotherapies for depression, PTSD, and opioid use disorder.

You may not qualify if:

• clinics with less than 12 months of data in 2018
• clinics already involved in Office of Veterans Access to Care (OVACS) quality improvement program at baseline.
• clinics where the VA Cerner electronic health record (EHR) implementation rollout will occur during the project period (Veterans Integrated Services Networks (VISNs) 20, 21 ,22, and 7)
• clinics who serve less than 122 unique patients each month on average
• clinics without an onsite multidisciplinary team of mental health or addiction service providers (minimum required: 1 psychiatrist, 1 psychologist, 1 social worker onsite)

Sponsors & Collaborators

• Palo Alto Veterans Institute for Research: lead

Study Sites (1)

VA Palo Alto Health Care System

Palo Alto, California, 94304, United States

Location

Related Publications (188)

• Elbogen EB, Wagner HR, Johnson SC, Kinneer P, Kang H, Vasterling JJ, Timko C, Beckham JC. Are Iraq and Afghanistan veterans using mental health services? New data from a national random-sample survey. Psychiatr Serv. 2013 Feb 1;64(2):134-41. doi: 10.1176/appi.ps.004792011.

  PMID: 23475498 BACKGROUND

• Hermes ED, Hoff R, Rosenheck RA. Sources of the increasing number of Vietnam era veterans with a diagnosis of PTSD using VHA services. Psychiatr Serv. 2014 Jun 1;65(6):830-2. doi: 10.1176/appi.ps.201300232.

  PMID: 24733559 BACKGROUND

• Mott JM, Grubbs KM, Sansgiry S, Fortney JC, Cully JA. Psychotherapy Utilization Among Rural and Urban Veterans From 2007 to 2010. J Rural Health. 2015 Summer;31(3):235-43. doi: 10.1111/jrh.12099. Epub 2014 Dec 3.

  PMID: 25471067 BACKGROUND

• Hankin CS, Spiro A 3rd, Miller DR, Kazis L. Mental disorders and mental health treatment among U.S. Department of Veterans Affairs outpatients: the Veterans Health Study. Am J Psychiatry. 1999 Dec;156(12):1924-30. doi: 10.1176/ajp.156.12.1924.

  PMID: 10588406 BACKGROUND

• Hoggatt KJ, Williams EC, Der-Martirosian C, Yano EM, Washington DL. National prevalence and correlates of alcohol misuse in women veterans. J Subst Abuse Treat. 2015 May;52:10-6. doi: 10.1016/j.jsat.2014.12.003. Epub 2014 Dec 19.

  PMID: 25661517 BACKGROUND

• Fulton JJ, Calhoun PS, Wagner HR, Schry AR, Hair LP, Feeling N, Elbogen E, Beckham JC. The prevalence of posttraumatic stress disorder in Operation Enduring Freedom/Operation Iraqi Freedom (OEF/OIF) Veterans: a meta-analysis. J Anxiety Disord. 2015 Apr;31:98-107. doi: 10.1016/j.janxdis.2015.02.003. Epub 2015 Feb 19.

  PMID: 25768399 BACKGROUND

• Karlin BE, Cross G. From the laboratory to the therapy room: National dissemination and implementation of evidence-based psychotherapies in the U.S. Department of Veterans Affairs Health Care System. Am Psychol. 2014 Jan;69(1):19-33. doi: 10.1037/a0033888. Epub 2013 Sep 2.

  PMID: 24001035 BACKGROUND

• Karlin BE, Brown GK, Trockel M, Cunning D, Zeiss AM, Taylor CB. National dissemination of cognitive behavioral therapy for depression in the Department of Veterans Affairs health care system: therapist and patient-level outcomes. J Consult Clin Psychol. 2012 Oct;80(5):707-18. doi: 10.1037/a0029328. Epub 2012 Jul 23.

  PMID: 22823859 BACKGROUND

• Ruzek JI, Karlin BE, & Zeiss AM. Implementation of Evidence-Based Psychological Treatments in the Veterans Health Administration. In: McHugh RK, Barlow DH, eds. Dissemination of evidence-based psychological treatments. New York, NY: Oxford University Press. , 2012.

  BACKGROUND

• Eftekhari A, Ruzek JI, Crowley JJ, Rosen CS, Greenbaum MA, Karlin BE. Effectiveness of national implementation of prolonged exposure therapy in Veterans Affairs care. JAMA Psychiatry. 2013 Sep;70(9):949-55. doi: 10.1001/jamapsychiatry.2013.36.

  PMID: 23863892 BACKGROUND

• Watts BV, Shiner B, Zubkoff L, Carpenter-Song E, Ronconi JM, Coldwell CM. Implementation of evidence-based psychotherapies for posttraumatic stress disorder in VA specialty clinics. Psychiatr Serv. 2014 May 1;65(5):648-53. doi: 10.1176/appi.ps.201300176.

  PMID: 24430622 BACKGROUND

• Department of Defense, & Department of Veterans Affairs. The management of MDD Working Group. VA/DOD clinical practice guideline for management of major depressive disorder (MDD). , 2009.

  BACKGROUND

• Department of Veterans Affairs, & Department of Defense. VA/DoD Clinical practice guideline for the management of post-traumatic stress. , 2010.

  BACKGROUND

• Department of Veterans Affairs, & Department of Defense. VA/DoD Clinical practice guideline for the management of substance use disorders. , 2009.

  BACKGROUND

• Lemke S, Boden MT, Kearney LK, Krahn DD, Neuman MJ, Schmidt EM, Trafton JA. Measurement-based management of mental health quality and access in VHA: SAIL mental health domain. Psychol Serv. 2017 Feb;14(1):1-12. doi: 10.1037/ser0000097.

  PMID: 28134552 BACKGROUND

• Harris AH, Humphreys K, Bowe T, Kivlahan DR, Finney JW. Measuring the quality of substance use disorder treatment: evaluating the validity of the Department of Veterans Affairs continuity of care performance measure. J Subst Abuse Treat. 2009 Apr;36(3):294-305. doi: 10.1016/j.jsat.2008.05.011. Epub 2008 Oct 5.

  PMID: 18835678 BACKGROUND

• Rosen CS, Matthieu MM, Wiltsey Stirman S, Cook JM, Landes S, Bernardy NC, Chard KM, Crowley J, Eftekhari A, Finley EP, Hamblen JL, Harik JM, Kehle-Forbes SM, Meis LA, Osei-Bonsu PE, Rodriguez AL, Ruggiero KJ, Ruzek JI, Smith BN, Trent L, Watts BV. A Review of Studies on the System-Wide Implementation of Evidence-Based Psychotherapies for Posttraumatic Stress Disorder in the Veterans Health Administration. Adm Policy Ment Health. 2016 Nov;43(6):957-977. doi: 10.1007/s10488-016-0755-0.

  PMID: 27474040 BACKGROUND

• Seal KH, Maguen S, Cohen B, Gima KS, Metzler TJ, Ren L, Bertenthal D, Marmar CR. VA mental health services utilization in Iraq and Afghanistan veterans in the first year of receiving new mental health diagnoses. J Trauma Stress. 2010 Feb;23(1):5-16. doi: 10.1002/jts.20493.

  PMID: 20146392 BACKGROUND

• Mott JM, Mondragon S, Hundt NE, Beason-Smith M, Grady RH, Teng EJ. Characteristics of U.S. veterans who begin and complete prolonged exposure and cognitive processing therapy for PTSD. J Trauma Stress. 2014 Jun;27(3):265-73. doi: 10.1002/jts.21927.

  PMID: 24948535 BACKGROUND

• Harpaz-Rotem I, Rosenheck RA. Serving those who served: retention of newly returning veterans from Iraq and Afghanistan in mental health treatment. Psychiatr Serv. 2011 Jan;62(1):22-7. doi: 10.1176/ps.62.1.pss6201_0022.

  PMID: 21209295 BACKGROUND

• Lin LA, Bohnert AS, Ilgen MA, Pfeiffer PN, Ganoczy D, Blow FC. Outpatient Provider Contact Prior to Unintentional Opioid Overdose Among VHA Service Users. Psychiatr Serv. 2015 Nov;66(11):1149-54. doi: 10.1176/appi.ps.201400194. Epub 2015 Jul 1.

  PMID: 26129993 BACKGROUND

• Harris AH, Bowe T, Del Re AC, Finlay AK, Oliva E, Myrick HL, Rubinsky AD. Extended release naltrexone for alcohol use disorders: quasi-experimental effects on mortality and subsequent detoxification episodes. Alcohol Clin Exp Res. 2015 Jan;39(1):79-83. doi: 10.1111/acer.12597.

  PMID: 25623408 BACKGROUND

• Degenhardt L, Bucello C, Mathers B, Briegleb C, Ali H, Hickman M, McLaren J. Mortality among regular or dependent users of heroin and other opioids: a systematic review and meta-analysis of cohort studies. Addiction. 2011 Jan;106(1):32-51. doi: 10.1111/j.1360-0443.2010.03140.x. Epub 2010 Nov 4.

  PMID: 21054613 BACKGROUND

• Kaplan MS, Huguet N, McFarland BH, Newsom JT. Suicide among male veterans: a prospective population-based study. J Epidemiol Community Health. 2007 Jul;61(7):619-24. doi: 10.1136/jech.2006.054346. Erratum In: J Epidemiol Community Health. 2007 Aug;61(8):751.

  PMID: 17568055 BACKGROUND

• Desai RA, Dausey DJ, Rosenheck RA. Mental health service delivery and suicide risk: the role of individual patient and facility factors. Am J Psychiatry. 2005 Feb;162(2):311-8. doi: 10.1176/appi.ajp.162.2.311.

  PMID: 15677596 BACKGROUND

• Gradus JL, Suvak MK, Wisco BE, Marx BP, Resick PA. Treatment of posttraumatic stress disorder reduces suicidal ideation. Depress Anxiety. 2013 Oct;30(10):1046-53. doi: 10.1002/da.22117. Epub 2013 May 1.

  PMID: 23636925 BACKGROUND

• Hysong SJ. Meta-analysis: audit and feedback features impact effectiveness on care quality. Med Care. 2009 Mar;47(3):356-63. doi: 10.1097/MLR.0b013e3181893f6b.

  PMID: 19194332 BACKGROUND

• Foy R, Eccles MP, Jamtvedt G, Young J, Grimshaw JM, Baker R. What do we know about how to do audit and feedback? Pitfalls in applying evidence from a systematic review. BMC Health Serv Res. 2005 Jul 13;5:50. doi: 10.1186/1472-6963-5-50.

  PMID: 16011811 BACKGROUND

• Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, O'Brien MA, Johansen M, Grimshaw J, Oxman AD. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012 Jun 13;2012(6):CD000259. doi: 10.1002/14651858.CD000259.pub3.

  PMID: 22696318 BACKGROUND

• Flottorp SA, Jamtvedt G, Gibis B, et al. Using audit and feedback to health professionals to improve the quality and safety of health care. World Health Organ. 2010.

  BACKGROUND

• Ivers NM, Sales A, Colquhoun H, Michie S, Foy R, Francis JJ, Grimshaw JM. No more 'business as usual' with audit and feedback interventions: towards an agenda for a reinvigorated intervention. Implement Sci. 2014 Jan 17;9:14. doi: 10.1186/1748-5908-9-14.

  PMID: 24438584 BACKGROUND

• Colquhoun HL, Carroll K, Eva KW, Grimshaw JM, Ivers N, Michie S, Sales A, Brehaut JC. Advancing the literature on designing audit and feedback interventions: identifying theory-informed hypotheses. Implement Sci. 2017 Sep 29;12(1):117. doi: 10.1186/s13012-017-0646-0.

  PMID: 28962632 BACKGROUND

• Colquhoun H, Michie S, Sales A, Ivers N, Grimshaw JM, Carroll K, Chalifoux M, Eva K, Brehaut J. Reporting and design elements of audit and feedback interventions: a secondary review. BMJ Qual Saf. 2017 Jan;26(1):54-60. doi: 10.1136/bmjqs-2015-005004. Epub 2016 Jan 25.

  PMID: 26811541 BACKGROUND

• Landis-Lewis Z, Brehaut JC, Hochheiser H, Douglas GP, Jacobson RS. Computer-supported feedback message tailoring: theory-informed adaptation of clinical audit and feedback for learning and behavior change. Implement Sci. 2015 Jan 21;10:12. doi: 10.1186/s13012-014-0203-z.

  PMID: 25603806 BACKGROUND

• Atkinson JA, Wells R, Page A, Dominello A, Haines M, Wilson A. Applications of system dynamics modelling to support health policy. Public Health Res Pract. 2015 Jul 9;25(3):e2531531. doi: 10.17061/phrp2531531.

  PMID: 26243490 BACKGROUND

• Diehl E, & Sterman JD. Effects of feedback complexity on dynamic decision making. Organ Behav Hum Decis Process. 1995;62:198-215.

  BACKGROUND

• Sterman JD. Modeling managerial behavior: Misperceptions of feedback in a dynamic decision making experiment. Manag Sci. 1989;35:321-339.

  BACKGROUND

• Cronin MA, Gonzalez C, & Sterman JD. Why don't well-educated adults understand accumulation? A challenge to researchers, educators, and citizens. Organ Behav Hum Decis Process. 2009;108:116-130.

  BACKGROUND

• Sterman JD. Learning from evidence in a complex world. Am J Public Health. 2006 Mar;96(3):505-14. doi: 10.2105/AJPH.2005.066043. Epub 2006 Jan 31.

  PMID: 16449579 BACKGROUND

• Sterman JD. Learning in and about complex systems. Syst Dyn Rev. 1994;10:291-330.

  BACKGROUND

• Andersen DF, Vennix JA, Richardson GP, et al. Group model building: Problem structing, policy simulation and decision support. J Oper Res Soc. 2007;:691-694.

  BACKGROUND

• Vennix JAM. Group model building: facilitating team learning using system dynamics. Chichester ; New York: J. Wiley, 1996: 1-297.

  BACKGROUND

• Rouwette EAJA, Vennix JAM, & Mullekom T van. Group model building effectiveness: A review of assessment studies. Syst Dyn Rev. 2002;18:5-45.

  BACKGROUND

• Bendoly E. System dynamics understanding in projects: Information sharing, psychological safety, and performance effects. Prod Oper Manag. 2014;23:1352-1369.

  BACKGROUND

• Sterman JD. Does formal system dynamics training improve people's understanding of accumulation? Syst Dyn Rev. 2010;26:316-334.

  BACKGROUND

• Simon HA. Bounded rationality and organizational learning. Organ Sci. 1991;2:125-134.

  BACKGROUND

• Rahmandad H, Repenning N, & Sterman J. Effects of feedback delay on learning. Syst Dyn Rev. 2009;25:309-338.

  BACKGROUND

• Zimmerman L, Lounsbury DW, Rosen CS, Kimerling R, Trafton JA, Lindley SE. Participatory System Dynamics Modeling: Increasing Stakeholder Engagement and Precision to Improve Implementation Planning in Systems. Adm Policy Ment Health. 2016 Nov;43(6):834-849. doi: 10.1007/s10488-016-0754-1.

  PMID: 27480546 BACKGROUND

• Oliva R. Model calibration as a testing strategy for system dynamics models. Eur J Oper Res. 2003;151:552-568.

  BACKGROUND

• Trafton JA, Greenberg G, Harris AH, Tavakoli S, Kearney L, McCarthy J, Blow F, Hoff R, Schohn M. VHA mental health information system: applying health information technology to monitor and facilitate implementation of VHA Uniform Mental Health Services Handbook requirements. Med Care. 2013 Mar;51(3 Suppl 1):S29-36. doi: 10.1097/MLR.0b013e31827da836.

  PMID: 23407008 BACKGROUND

• Barlas Y. Formal aspects of model validity and validation in system dynamics. Syst Dyn Rev. 1996;12:183-210.

  BACKGROUND

• Hirsch G, Homer J, Trogdon J, Wile K, Orenstein D. Using simulation to compare 4 categories of intervention for reducing cardiovascular disease risks. Am J Public Health. 2014 Jul;104(7):1187-95. doi: 10.2105/AJPH.2013.301816. Epub 2014 May 15.

  PMID: 24832142 BACKGROUND

• Lich KH, Tian Y, Beadles CA, Williams LS, Bravata DM, Cheng EM, Bosworth HB, Homer JB, Matchar DB. Strategic planning to reduce the burden of stroke among veterans: using simulation modeling to inform decision making. Stroke. 2014 Jul;45(7):2078-84. doi: 10.1161/STROKEAHA.114.004694. Epub 2014 Jun 12.

  PMID: 24923722 BACKGROUND

• Forrester, J.W. The model versus a modeling process. Syst Dyn Rev. 1985;:133-134.

  BACKGROUND

• Derickson R, Fishman J, Osatuke K, Teclaw R, Ramsel D. Psychological safety and error reporting within Veterans Health Administration hospitals. J Patient Saf. 2015 Mar;11(1):60-6. doi: 10.1097/PTS.0000000000000082.

  PMID: 24583957 BACKGROUND

• Edmondson A. Psychological safety and learning behavior in work teams. Adm Sci Q. 1999;44:350-383.

  BACKGROUND

• Campbell MJ, & Walters SJ. How to Design, Analyise and Report Cluster Randomised Trials in Medicine and Health Related Research. : Wiley, 2014: 1-264.

  BACKGROUND

• Forrester JW. System dynamics, systems thinking, and soft OR. Syst Dyn Rev. 1994;10:245-256.

  BACKGROUND

• Forrester JW. Some basic concepts in system dynamics. Sloan Sch Manag Mass Inst Technol Camb MA. 2009.

  BACKGROUND

• Hong G. Ratio of mediator probability weighting for estimating natural direct and indirect effects. In: Proceedings of the American Statistical Association, Biometrics Section. : American Statistical Association Alexandria, VA, 2010: 2401-2415.

  BACKGROUND

• Hong G, Deutsch J, & Hill HD. Ratio-of-Mediator-Probability Weighting for Causal Mediation Analysis in the Presence of Treatment-by-Mediator Interaction. J Educ Behav Stat. 2015;40:307-340.

  BACKGROUND

• Qin X, & Hong G. Causal mediation analysis in multi-site trials: An application of ratio-of-mediatorprobability weighting to the Head Start Impact Study. JSM Proc Soc Stat Sect. 2014;:912-926.

  BACKGROUND

• Homer JB. Partial-model testing as a validation tool for system dynamics (1983). Syst Dyn Rev. 2012;28:281-294.

  BACKGROUND

• Rahmandad H, & Sterman JD. Reporting guidelines for simulation-based research in social sciences. Syst Dyn Rev. 2012;28:396-411.

  BACKGROUND

• Singer SJ, Moore SC, Meterko M, Williams S. Development of a short-form Learning Organization Survey: the LOS-27. Med Care Res Rev. 2012 Aug;69(4):432-59. doi: 10.1177/1077558712448135. Epub 2012 May 25.

  PMID: 22635274 BACKGROUND

• Batorowicz B, Shepherd TA. Measuring the quality of transdisciplinary teams. J Interprof Care. 2008 Dec;22(6):612-20. doi: 10.1080/13561820802303664.

  PMID: 19012141 BACKGROUND

• Dolan ED, Mohr D, Lempa M, Joos S, Fihn SD, Nelson KM, Helfrich CD. Using a single item to measure burnout in primary care staff: a psychometric evaluation. J Gen Intern Med. 2015 May;30(5):582-7. doi: 10.1007/s11606-014-3112-6. Epub 2014 Dec 2.

  PMID: 25451989 BACKGROUND

• Helfrich CD, Simonetti JA, Clinton WL, Wood GB, Taylor L, Schectman G, Stark R, Rubenstein LV, Fihn SD, Nelson KM. The Association of Team-Specific Workload and Staffing with Odds of Burnout Among VA Primary Care Team Members. J Gen Intern Med. 2017 Jul;32(7):760-766. doi: 10.1007/s11606-017-4011-4. Epub 2017 Feb 23.

  PMID: 28233221 BACKGROUND

• Nelson KM, Helfrich C, Sun H, Hebert PL, Liu CF, Dolan E, Taylor L, Wong E, Maynard C, Hernandez SE, Sanders W, Randall I, Curtis I, Schectman G, Stark R, Fihn SD. Implementation of the patient-centered medical home in the Veterans Health Administration: associations with patient satisfaction, quality of care, staff burnout, and hospital and emergency department use. JAMA Intern Med. 2014 Aug;174(8):1350-8. doi: 10.1001/jamainternmed.2014.2488.

  PMID: 25055197 BACKGROUND

• Helfrich CD, Dolan ED, Simonetti J, Reid RJ, Joos S, Wakefield BJ, Schectman G, Stark R, Fihn SD, Harvey HB, Nelson K. Elements of team-based care in a patient-centered medical home are associated with lower burnout among VA primary care employees. J Gen Intern Med. 2014 Jul;29 Suppl 2(Suppl 2):S659-66. doi: 10.1007/s11606-013-2702-z.

  PMID: 24715396 BACKGROUND

• Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, Boynton MH, Halko H. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci. 2017 Aug 29;12(1):108. doi: 10.1186/s13012-017-0635-3.

  PMID: 28851459 BACKGROUND

• Sweeney LB, & Sterman JD. Bathtub dynamics: Initial results of a systems thinking inventory. Syst Dyn Rev. 2000;16:249-286.

  BACKGROUND

• Liberati EG, Ruggiero F, Galuppo L, Gorli M, Gonzalez-Lorenzo M, Maraldi M, Ruggieri P, Polo Friz H, Scaratti G, Kwag KH, Vespignani R, Moja L. What hinders the uptake of computerized decision support systems in hospitals? A qualitative study and framework for implementation. Implement Sci. 2017 Sep 15;12(1):113. doi: 10.1186/s13012-017-0644-2.

  PMID: 28915822 BACKGROUND

• Centers for Medicare & Medicaid Services Alliance to Modernize Healthcare (CAMH). Independent Assessment of the Health Care Delivery Systems and Management Processes of the Department of Veterans Affairs (Volume 1: Integrated Report).

  BACKGROUND

• Department of Veterans Affairs. Uniform mental health services in VA medical centers and clinics. Washington DC: Veterans Health Administration, 2008.

  BACKGROUND

• Powers MB, Halpern JM, Ferenschak MP, Gillihan SJ, Foa EB. A meta-analytic review of prolonged exposure for posttraumatic stress disorder. Clin Psychol Rev. 2010 Aug;30(6):635-41. doi: 10.1016/j.cpr.2010.04.007. Epub 2010 May 2.

  PMID: 20546985 BACKGROUND

• Bradley R, Greene J, Russ E, Dutra L, Westen D. A multidimensional meta-analysis of psychotherapy for PTSD. Am J Psychiatry. 2005 Feb;162(2):214-27. doi: 10.1176/appi.ajp.162.2.214.

  PMID: 15677582 BACKGROUND

• Powers MB, Zum Vorde Sive Vording MB, Emmelkamp PM. Acceptance and commitment therapy: a meta-analytic review. Psychother Psychosom. 2009;78(2):73-80. doi: 10.1159/000190790. Epub 2009 Jan 14.

  PMID: 19142046 BACKGROUND

• Walser RD, Karlin BE, Trockel M, Mazina B, Barr Taylor C. Training in and implementation of Acceptance and Commitment Therapy for depression in the Veterans Health Administration: therapist and patient outcomes. Behav Res Ther. 2013 Sep;51(9):555-63. doi: 10.1016/j.brat.2013.05.009. Epub 2013 Jun 3.

  PMID: 23851161 BACKGROUND

• Stewart MO, Raffa SD, Steele JL, Miller SA, Clougherty KF, Hinrichsen GA, Karlin BE. National dissemination of interpersonal psychotherapy for depression in veterans: therapist and patient-level outcomes. J Consult Clin Psychol. 2014 Dec;82(6):1201-6. doi: 10.1037/a0037410. Epub 2014 Jul 21.

  PMID: 25045906 BACKGROUND

• Cuijpers P, Geraedts AS, van Oppen P, Andersson G, Markowitz JC, van Straten A. Interpersonal psychotherapy for depression: a meta-analysis. Am J Psychiatry. 2011 Jun;168(6):581-92. doi: 10.1176/appi.ajp.2010.10101411. Epub 2011 Mar 1.

  PMID: 21362740 BACKGROUND

• Fournier JC, DeRubeis RJ, Hollon SD, Dimidjian S, Amsterdam JD, Shelton RC, Fawcett J. Antidepressant drug effects and depression severity: a patient-level meta-analysis. JAMA. 2010 Jan 6;303(1):47-53. doi: 10.1001/jama.2009.1943.

  PMID: 20051569 BACKGROUND

• Knudsen HK, Roman PM. The Diffusion of Acamprosate for the Treatment of Alcohol Use Disorder: Results From a National Longitudinal Study. J Subst Abuse Treat. 2016 Mar;62:62-7. doi: 10.1016/j.jsat.2015.10.005. Epub 2015 Dec 1.

  PMID: 26689318 BACKGROUND

• Knudsen HK, Roman PM. Service delivery and pharmacotherapy for alcohol use disorder in the era of health reform: Data from a national sample of treatment organizations. Subst Abus. 2016;37(1):230-7. doi: 10.1080/08897077.2015.1028699. Epub 2015 Apr 20.

  PMID: 25893539 BACKGROUND

• Pettinati HM, O'Brien CP, Rabinowitz AR, Wortman SP, Oslin DW, Kampman KM, Dackis CA. The status of naltrexone in the treatment of alcohol dependence: specific effects on heavy drinking. J Clin Psychopharmacol. 2006 Dec;26(6):610-25. doi: 10.1097/01.jcp.0000245566.52401.20.

  PMID: 17110818 BACKGROUND

• Anton RF, O'Malley SS, Ciraulo DA, Cisler RA, Couper D, Donovan DM, Gastfriend DR, Hosking JD, Johnson BA, LoCastro JS, Longabaugh R, Mason BJ, Mattson ME, Miller WR, Pettinati HM, Randall CL, Swift R, Weiss RD, Williams LD, Zweben A; COMBINE Study Research Group. Combined pharmacotherapies and behavioral interventions for alcohol dependence: the COMBINE study: a randomized controlled trial. JAMA. 2006 May 3;295(17):2003-17. doi: 10.1001/jama.295.17.2003.

  PMID: 16670409 BACKGROUND

• Barnett PG, Rodgers JH, Bloch DA. A meta-analysis comparing buprenorphine to methadone for treatment of opiate dependence. Addiction. 2001 May;96(5):683-90. doi: 10.1046/j.1360-0443.2001.9656834.x.

  PMID: 11331027 BACKGROUND

• West SL, O'Neal KK, Graham CW. A meta-analysis comparing the effectiveness of buprenorphine and methadone. J Subst Abuse. 2000;12(4):405-14. doi: 10.1016/s0899-3289(01)00054-2.

  PMID: 11452842 BACKGROUND

• Hysong SJ, Kell HJ, Petersen LA, Campbell BA, Trautner BW. Theory-based and evidence-based design of audit and feedback programmes: examples from two clinical intervention studies. BMJ Qual Saf. 2017 Apr;26(4):323-334. doi: 10.1136/bmjqs-2015-004796. Epub 2016 Jun 10.

  PMID: 27288054 BACKGROUND

• Hysong SJ, Best RG, Pugh JA. Audit and feedback and clinical practice guideline adherence: making feedback actionable. Implement Sci. 2006 Apr 28;1:9. doi: 10.1186/1748-5908-1-9.

  PMID: 16722539 BACKGROUND

• Hysong SJ, Teal CR, Khan MJ, Haidet P. Improving quality of care through improved audit and feedback. Implement Sci. 2012 May 18;7:45. doi: 10.1186/1748-5908-7-45.

  PMID: 22607640 BACKGROUND

• Shiner B, D'Avolio LW, Nguyen TM, Zayed MH, Young-Xu Y, Desai RA, Schnurr PP, Fiore LD, Watts BV. Measuring use of evidence based psychotherapy for posttraumatic stress disorder. Adm Policy Ment Health. 2013 Jul;40(4):311-8. doi: 10.1007/s10488-012-0421-0.

  PMID: 22535469 BACKGROUND

• Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999 Sep;89(9):1322-7. doi: 10.2105/ajph.89.9.1322.

  PMID: 10474547 BACKGROUND

• Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, Griffey R, Hensley M. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011 Mar;38(2):65-76. doi: 10.1007/s10488-010-0319-7.

  PMID: 20957426 BACKGROUND

• Brownson RC, Colditz GA, & Proctor EK. Dissemination and Implementation Research in Health: Translating Science to Practice. Oxford, UK: Oxford, 2012.

  BACKGROUND

• Tabak RG, Khoong EC, Chambers DA, Brownson RC. Bridging research and practice: models for dissemination and implementation research. Am J Prev Med. 2012 Sep;43(3):337-50. doi: 10.1016/j.amepre.2012.05.024.

  PMID: 22898128 BACKGROUND

• Tabak RG, Khoong EC, Chambers D, et al. Models in dissemination and implementation research: useful tools in public health services and systems research. Front Public Health Serv Syst Res. 2013;2:8.

  BACKGROUND

• Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009 Aug 7;4:50. doi: 10.1186/1748-5908-4-50.

  PMID: 19664226 BACKGROUND

• Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, Proctor EK, Kirchner JE. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015 Feb 12;10:21. doi: 10.1186/s13012-015-0209-1.

  PMID: 25889199 BACKGROUND

• Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. 2013 Oct 2;8:117. doi: 10.1186/1748-5908-8-117.

  PMID: 24088228 BACKGROUND

• Breuer E, Lee L, De Silva M, Lund C. Using theory of change to design and evaluate public health interventions: a systematic review. Implement Sci. 2016 May 6;11:63. doi: 10.1186/s13012-016-0422-6.

  PMID: 27153985 BACKGROUND

• Schierhout G, Hains J, Si D, Kennedy C, Cox R, Kwedza R, O'Donoghue L, Fittock M, Brands J, Lonergan K, Dowden M, Bailie R. Evaluating the effectiveness of a multifaceted, multilevel continuous quality improvement program in primary health care: developing a realist theory of change. Implement Sci. 2013 Oct 8;8:119. doi: 10.1186/1748-5908-8-119.

  PMID: 24098940 BACKGROUND

• Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015 Apr 21;10:53. doi: 10.1186/s13012-015-0242-0.

  PMID: 25895742 BACKGROUND

• Sales A, Smith J, Curran G, Kochevar L. Models, strategies, and tools. Theory in implementing evidence-based findings into health care practice. J Gen Intern Med. 2006 Feb;21 Suppl 2(Suppl 2):S43-9. doi: 10.1111/j.1525-1497.2006.00362.x.

  PMID: 16637960 BACKGROUND

• Damschroder LJ, & Lowery JC. Efficient Synthesis: Using Qualitative Comparative Analysis (QCA) and the CFIR across Diverse Studies. 2015.

  BACKGROUND

• Baumgartner M, & Thiem A. Often Trusted but Never (Properly) Tested: Evaluating Qualitative Comparative Analysis. Sociol Methods Res. 2017;:4912411770148.

  BACKGROUND

• Hug S. Qualitative Comparative Analysis: How Inductive Use and Measurement Error Lead to Problematic Inference. Polit Anal. 2013;21:252-265.

  BACKGROUND

• Krogslund C, Choi DD, & Poertner M. Fuzzy Sets on Shaky Ground: Parameter Sensitivity and Confirmation Bias in fsQCA. Polit Anal. 2015;23:21-41.

  BACKGROUND

• Braumoeller B. QCAfalsePositive: Tests for Type I Error in Qualitative Comparative Analysis (QCA). 2015.

  BACKGROUND

• Powell BJ, Proctor EK, Glass JE. A Systematic Review of Strategies for Implementing Empirically Supported Mental Health Interventions. Res Soc Work Pract. 2014 Mar 1;24(2):192-212. doi: 10.1177/1049731513505778.

  PMID: 24791131 BACKGROUND

• Williams NJ. Multilevel Mechanisms of Implementation Strategies in Mental Health: Integrating Theory, Research, and Practice. Adm Policy Ment Health. 2016 Sep;43(5):783-798. doi: 10.1007/s10488-015-0693-2.

  PMID: 26474761 BACKGROUND

• Senge P. The Fifth Discipline. : Currency/Doubleday, 1990.

  BACKGROUND

• Sweeney LB, & Sterman JD. Bathtub dynamics: initial results of a systems thinking inventory. Syst Dyn Rev. 2000;16:249-286.

  BACKGROUND

• Hysong SJ, Knox MK, Haidet P. Examining clinical performance feedback in Patient-Aligned Care Teams. J Gen Intern Med. 2014 Jul;29 Suppl 2(Suppl 2):S667-74. doi: 10.1007/s11606-013-2707-7.

  PMID: 24715398 BACKGROUND

• Hysong SJ, Thomas CL, Spitzmuller C, Amspoker AB, Woodard L, Modi V, Naik AD. Linking clinician interaction and coordination to clinical performance in Patient-Aligned Care Teams. Implement Sci. 2016 Jan 15;11:7. doi: 10.1186/s13012-015-0368-0.

  PMID: 26772972 BACKGROUND

• Richardson GP. Concept models in group model building: G. P. Richardson: Concept Models in Group Model Building. Syst Dyn Rev. 2013;29:42-55.

  BACKGROUND

• Richardson GP. Reflections on the foundations of system dynamics: Foundations of System Dynamics. Syst Dyn Rev. 2011;27:219-243.

  BACKGROUND

• Garcia HA, McGeary CA, Finley EP, et al. Burnout among psychiatrists in the Veterans Health Administration. Burn Res. 2015;2:108-114.

  BACKGROUND

• Garcia HA, Finley EP, Ketchum N, Jakupcak M, Dassori A, Reyes SC. A survey of perceived barriers and attitudes toward mental health care among OEF/OIF veterans at VA outpatient mental health clinics. Mil Med. 2014 Mar;179(3):273-8. doi: 10.7205/MILMED-D-13-00076.

  PMID: 24594461 BACKGROUND

• Garcia HA, Kelley LP, Rentz TO, et al. Pretreatment predictors of dropout from cognitive behavioral therapy for PTSD in Iraq and Afghanistan war veterans. Psychol Serv. 2011;8:1-11.

  BACKGROUND

• Senge, P.M. The fifth discipline: The art and practice of the learning organization. : Broadway Business, 2006.

  BACKGROUND

• Meadows DH. Thinking in systems: A primer. : Chelsea Green Publishing, 2012.

  BACKGROUND

• Senge PM, ed. The dance of change: the challenges of sustaining momentum in learning organizations, 1st ed. New York: Currency/Doubleday, 1999: 1-596.

  BACKGROUND

• Sterman JD. Business Dynamics: Systems Thinking and Modeling for a Complex World. : McGraw-Hill Education, 2000: 1-1008.

  BACKGROUND

• Rouwette EA, & Vennix JA. System dynamics and organizational interventions. Syst Res Behav Sci. 2006;23:451-466.

  BACKGROUND

• Akhnif E, Macq J, Idrissi Fakhreddine MO, Meessen B. Scoping literature review on the Learning Organisation concept as applied to the health system. Health Res Policy Syst. 2017 Mar 1;15(1):16. doi: 10.1186/s12961-017-0176-x.

  PMID: 28249608 BACKGROUND

• Morecroft J, & Sherman J. Modeling for learning organizations. Portland OR: Productivity Press, 1994.

  BACKGROUND

• Argyris C, & Schön DA. Organizational learning: A theory of action perspective. Reis. 1997;:345-348.

  BACKGROUND

• Forrester JW. Industrial Dynamics. Cambridge, MA: MIT Press, 1961.

  BACKGROUND

• Simon HA. Models of Man: Social and Rational-Mathematical Essays on Rational Human Behavior in a Social Setting. Oxford, UK: Wiley, 1957.

  BACKGROUND

• Huz S, Andersen DF, Richardson GP, et al. A framework for evaluating systems thinking interventions: an experimental approach to mental health system change. Syst Dyn Rev. 1997;13:149-169.

  BACKGROUND

• Evans JS. Dual-processing accounts of reasoning, judgment, and social cognition. Annu Rev Psychol. 2008;59:255-78. doi: 10.1146/annurev.psych.59.103006.093629.

  PMID: 18154502 BACKGROUND

• Evans JS. In two minds: dual-process accounts of reasoning. Trends Cogn Sci. 2003 Oct;7(10):454-9. doi: 10.1016/j.tics.2003.08.012.

  PMID: 14550493 BACKGROUND

• Evans JS, Stanovich KE. Dual-Process Theories of Higher Cognition: Advancing the Debate. Perspect Psychol Sci. 2013 May;8(3):223-41. doi: 10.1177/1745691612460685.

  PMID: 26172965 BACKGROUND

• Dolansky MA, Moore SM, Palmieri PA, Singh MK. Development and Validation of the Systems Thinking Scale. J Gen Intern Med. 2020 Aug;35(8):2314-2320. doi: 10.1007/s11606-020-05830-1. Epub 2020 Apr 27.

  PMID: 32342481 BACKGROUND

• Hovmand PS. Community Based System Dynamics. New York, NY: Springer New York, 2014.

  BACKGROUND

• Minkler M, & Wallerstein N. Community-Based Participatory Research for Health: From Process to Outcomes. : John Wiley & Sons, 2011: 1-758.

  BACKGROUND

• Case AD, Byrd R, Claggett E, DeVeaux S, Perkins R, Huang C, Sernyak MJ, Steiner JL, Cole R, LaPaglia DM, Bailey M, Buchanan C, Johnson A, Kaufman JS. Stakeholders' perspectives on community-based participatory research to enhance mental health services. Am J Community Psychol. 2014 Dec;54(3-4):397-408. doi: 10.1007/s10464-014-9677-8.

  PMID: 25245601 BACKGROUND

• Israel BA, Coombe CM, Cheezum RR, Schulz AJ, McGranaghan RJ, Lichtenstein R, Reyes AG, Clement J, Burris A. Community-based participatory research: a capacity-building approach for policy advocacy aimed at eliminating health disparities. Am J Public Health. 2010 Nov;100(11):2094-102. doi: 10.2105/AJPH.2009.170506. Epub 2010 Sep 23.

  PMID: 20864728 BACKGROUND

• Wallerstein NB, Duran B. Using community-based participatory research to address health disparities. Health Promot Pract. 2006 Jul;7(3):312-23. doi: 10.1177/1524839906289376. Epub 2006 Jun 7.

  PMID: 16760238 BACKGROUND

• Aarons GA. Measuring provider attitudes toward evidence-based practice: consideration of organizational context and individual differences. Child Adolesc Psychiatr Clin N Am. 2005 Apr;14(2):255-71, viii. doi: 10.1016/j.chc.2004.04.008.

  PMID: 15694785 BACKGROUND

• Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health. 2011 Jan;38(1):4-23. doi: 10.1007/s10488-010-0327-7.

  PMID: 21197565 BACKGROUND

• Aarons GA, Ehrhart MG, Farahnak LR, Sklar M. Aligning leadership across systems and organizations to develop a strategic climate for evidence-based practice implementation. Annu Rev Public Health. 2014;35:255-74. doi: 10.1146/annurev-publhealth-032013-182447.

  PMID: 24641560 BACKGROUND

• Aarons GA, Green AE, Trott E, Willging CE, Torres EM, Ehrhart MG, Roesch SC. The Roles of System and Organizational Leadership in System-Wide Evidence-Based Intervention Sustainment: A Mixed-Method Study. Adm Policy Ment Health. 2016 Nov;43(6):991-1008. doi: 10.1007/s10488-016-0751-4.

  PMID: 27439504 BACKGROUND

• Brimhall KC, Fenwick K, Farahnak LR, Hurlburt MS, Roesch SC, Aarons GA. Leadership, Organizational Climate, and Perceived Burden of Evidence-Based Practice in Mental Health Services. Adm Policy Ment Health. 2016 Sep;43(5):629-639. doi: 10.1007/s10488-015-0670-9.

  PMID: 26152770 BACKGROUND

• Senge P, & Forrester JW. Tests for building confidence in system dynamics models. 1980;14:209-228.

  BACKGROUND

• Rahmandad H, & Sterman JD. Reporting guidelines for simulation-based research in social sciences: Reporting Guidelines for Simulation-Based Research. Syst Dyn Rev. 2012;28:396-411.

  BACKGROUND

• Hovmand PS, Andersen DF, Rouwette E, et al. Group model-building "scripts" as a collaborative planning tool: Scripts as a collaborative planning tool. Syst Res Behav Sci. 2012;29:179-193.

  BACKGROUND

• Andersen D, & Richardson G. Scripts for group model building. Syst Dyn Rev. 1997;13:107-129.

  BACKGROUND

• Hovmand, Peter, S., et al. Scriptapedia 4.0.6. 2013.

  BACKGROUND

• McCarthy JT, Hocum CL, Albright RC, Rogers J, Gallaher EJ, Steensma DP, Gudgell SF, Bergstralh EJ, Dillon JC, Hickson LJ, Williams AW, Dingli D. Biomedical system dynamics to improve anemia control with darbepoetin alfa in long-term hemodialysis patients. Mayo Clin Proc. 2014 Jan;89(1):87-94. doi: 10.1016/j.mayocp.2013.10.022.

  PMID: 24388026 BACKGROUND

• Gallaher E, Steensma DP, Chrisope TR, et al. Individualized Medicine and Biophysical System Dynamics: An Example from Clinical Practice in End Stage Renal Disease. 2011.

  BACKGROUND

• Homer J. Levels of evidence in system dynamics modeling. Syst Dyn Rev. 2014;30:75-80.

  BACKGROUND

• Oliva R, & Sterman JD. Death spirals and virtuous cycles. In: Handbook of Service Science. : Springer, 2010: 321-358.

  BACKGROUND

• Snow G. Package "blockrand." 2015.

  BACKGROUND

• Holowka DW, Marx BP, Gates MA, Litman HJ, Ranganathan G, Rosen RC, Keane TM. PTSD diagnostic validity in Veterans Affairs electronic records of Iraq and Afghanistan veterans. J Consult Clin Psychol. 2014 Aug;82(4):569-579. doi: 10.1037/a0036347. Epub 2014 Apr 14.

  PMID: 24731235 BACKGROUND

• Emmons KM, Weiner B, Fernandez ME, Tu SP. Systems antecedents for dissemination and implementation: a review and analysis of measures. Health Educ Behav. 2012 Feb;39(1):87-105. doi: 10.1177/1090198111409748. Epub 2011 Jul 1.

  PMID: 21724933 BACKGROUND

• Rabin BA, Lewis CC, Norton WE, Neta G, Chambers D, Tobin JN, Brownson RC, Glasgow RE. Measurement resources for dissemination and implementation research in health. Implement Sci. 2016 Mar 22;11:42. doi: 10.1186/s13012-016-0401-y.

  PMID: 27000147 BACKGROUND

• Lewis CC, Fischer S, Weiner BJ, Stanick C, Kim M, Martinez RG. Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria. Implement Sci. 2015 Nov 4;10:155. doi: 10.1186/s13012-015-0342-x.

  PMID: 26537706 BACKGROUND

• Glasgow RE. What does it mean to be pragmatic? Pragmatic methods, measures, and models to facilitate research translation. Health Educ Behav. 2013 Jun;40(3):257-65. doi: 10.1177/1090198113486805.

  PMID: 23709579 BACKGROUND

• Glasgow RE, Riley WT. Pragmatic measures: what they are and why we need them. Am J Prev Med. 2013 Aug;45(2):237-43. doi: 10.1016/j.amepre.2013.03.010.

  PMID: 23867032 BACKGROUND

• Singh MK, Ogrinc G, Cox KR, Dolansky M, Brandt J, Morrison LJ, Harwood B, Petroski G, West A, Headrick LA. The Quality Improvement Knowledge Application Tool Revised (QIKAT-R). Acad Med. 2014 Oct;89(10):1386-91. doi: 10.1097/ACM.0000000000000456.

  PMID: 25119555 BACKGROUND

• Rosseel Y, Oberski D, Byrnes J, et al. Package "lavaan" Version 0.5-23.1097. 2017.

  BACKGROUND

• Shavelson RJ, & Webb NM. Generalizability theory: A primer. Newbury Park, CA: Sage., 1991.

  BACKGROUND

• Millsap RE, Kwok OM. Evaluating the impact of partial factorial invariance on selection in two populations. Psychol Methods. 2004 Mar;9(1):93-115. doi: 10.1037/1082-989X.9.1.93.

  PMID: 15053721 BACKGROUND

• Wolak M. Package "ICC" Version 2.3.0. 2015.

  BACKGROUND

• Bates D, Maechler M, Bolker B, et al. Package "lme4" Version 1.1-13. 2017.

  BACKGROUND

• Rotondi MA. Package "CRTSize" Version 1.0. 2015.

  BACKGROUND

• Rotondi MA, & Donner A. Sample Size Estimation in Cluster Randomized Educational Trials: An Empirical Bayes Approach. J Educ Behav Stat. 2009;34:229-237.

  BACKGROUND

• Ventana Systems Inc. Vensim@ Version 6.3. 2014.

  BACKGROUND

• Hyndman, R.J., & Khandakar, Y. Automatic time series forecasting: The forecast package for R. J Stat Softw. 2008;26.

  BACKGROUND

• Hyndman R, O'Hara-Wild Mi, Bergmeir C, et al. Package "forecast" Version 8.0. 2017.

  BACKGROUND

• Mott JM, Hundt NE, Sansgiry S, Mignogna J, Cully JA. Changes in psychotherapy utilization among veterans with depression, anxiety, and PTSD. Psychiatr Serv. 2014 Jan 1;65(1):106-12. doi: 10.1176/appi.ps.201300056.

  PMID: 24081213 BACKGROUND

• Oliva EM, Trafton JA, Harris AH, Gordon AJ. Trends in opioid agonist therapy in the Veterans Health Administration: is supply keeping up with demand? Am J Drug Alcohol Abuse. 2013 Mar;39(2):103-7. doi: 10.3109/00952990.2012.741167.

  PMID: 23421571 BACKGROUND

• Diaz M, Neuhauser D. Pasteur and parachutes: when statistical process control is better than a randomized controlled trial. Qual Saf Health Care. 2005 Apr;14(2):140-3. doi: 10.1136/qshc.2005.013763. No abstract available.

  PMID: 15805461 BACKGROUND

• Duclos A, Voirin N. The p-control chart: a tool for care improvement. Int J Qual Health Care. 2010 Oct;22(5):402-7. doi: 10.1093/intqhc/mzq037. Epub 2010 Jul 30.

  PMID: 20675711 BACKGROUND

• Ogrinc GS, & Headrick L. Fundamentals of health care improvement: A guide to improving your patients' care. : Joint Commission Resources, 2008.

  BACKGROUND

• Ogrinc G, Mooney SE, Estrada C, Foster T, Goldmann D, Hall LW, Huizinga MM, Liu SK, Mills P, Neily J, Nelson W, Pronovost PJ, Provost L, Rubenstein LV, Speroff T, Splaine M, Thomson R, Tomolo AM, Watts B. The SQUIRE (Standards for QUality Improvement Reporting Excellence) guidelines for quality improvement reporting: explanation and elaboration. Qual Saf Health Care. 2008 Oct;17 Suppl 1(Suppl_1):i13-32. doi: 10.1136/qshc.2008.029058.

  PMID: 18836062 BACKGROUND

• Buntin MB, Burke MF, Hoaglin MC, Blumenthal D. The benefits of health information technology: a review of the recent literature shows predominantly positive results. Health Aff (Millwood). 2011 Mar;30(3):464-71. doi: 10.1377/hlthaff.2011.0178.

  PMID: 21383365 BACKGROUND

• Raffa SD, Maciejewski ML, Zimmerman LE, Damschroder LJ, Estabrooks PA, Ackermann RT, Tsai AG, Histon T, Goldstein MG. A System-Level Approach to Overweight and Obesity in the Veterans Health Administration. J Gen Intern Med. 2017 Apr;32(Suppl 1):79-82. doi: 10.1007/s11606-016-3948-z.

  PMID: 28271428 BACKGROUND

• Kilbourne AM, Elwy AR, Sales AE, Atkins D. Accelerating Research Impact in a Learning Health Care System: VA's Quality Enhancement Research Initiative in the Choice Act Era. Med Care. 2017 Jul;55 Suppl 7 Suppl 1(7 Suppl 1):S4-S12. doi: 10.1097/MLR.0000000000000683. Erratum In: Med Care. 2019 Nov;57(11):920. doi: 10.1097/MLR.0000000000001230.

  PMID: 27997456 BACKGROUND

• Mabry PL, Olster DH, Morgan GD, Abrams DB. Interdisciplinarity and systems science to improve population health: a view from the NIH Office of Behavioral and Social Sciences Research. Am J Prev Med. 2008 Aug;35(2 Suppl):S211-24. doi: 10.1016/j.amepre.2008.05.018.

  PMID: 18619402 BACKGROUND

• Mabry PL, Milstein B, Abraido-Lanza AF, Livingood WC, Allegrante JP. Opening a window on systems science research in health promotion and public health. Health Educ Behav. 2013 Oct;40(1 Suppl):5S-8S. doi: 10.1177/1090198113503343. No abstract available.

  PMID: 24084401 BACKGROUND

• Luke DA, Stamatakis KA. Systems science methods in public health: dynamics, networks, and agents. Annu Rev Public Health. 2012 Apr;33:357-76. doi: 10.1146/annurev-publhealth-031210-101222. Epub 2012 Jan 3.

  PMID: 22224885 BACKGROUND

• Moore GF, Evans RE. What theory, for whom and in which context? Reflections on the application of theory in the development and evaluation of complex population health interventions. SSM Popul Health. 2017 Dec;3:132-135. doi: 10.1016/j.ssmph.2016.12.005.

  PMID: 29302610 BACKGROUND

• Monks T. Operational research as implementation science: definitions, challenges and research priorities. Implement Sci. 2016 Jun 6;11(1):81. doi: 10.1186/s13012-016-0444-0.

  PMID: 27268021 BACKGROUND

• May CR, Johnson M, Finch T. Implementation, context and complexity. Implement Sci. 2016 Oct 19;11(1):141. doi: 10.1186/s13012-016-0506-3.

  PMID: 27756414 BACKGROUND

• Brainard J, Hunter PR. Do complexity-informed health interventions work? A scoping review. Implement Sci. 2016 Sep 20;11(1):127. doi: 10.1186/s13012-016-0492-5.

  PMID: 27647152 BACKGROUND

MeSH Terms

Conditions

Stress Disorders, Post-Traumatic; Depression; Alcoholism; Opioid-Related Disorders; Behavior, Addictive; Psychological Well-Being

Condition Hierarchy (Ancestors)

Stress Disorders, Traumatic; Trauma and Stressor Related Disorders; Mental Disorders; Behavioral Symptoms; Behavior; Alcohol-Related Disorders; Substance-Related Disorders; Chemically-Induced Disorders; Narcotic-Related Disorders; Compulsive Behavior; Impulsive Behavior; Personal Satisfaction

Study Officials

• Lindsey E Zimmerman, PhD

  National Center for PTSD, Dissemination & Training Division

  PRINCIPAL INVESTIGATOR

Study Design

• Study Type: interventional
• Phase: not applicable
• Allocation: RANDOMIZED
• Masking: NONE
• Purpose: HEALTH SERVICES RESEARCH
• Intervention Model: PARALLEL
• Sponsor Type: OTHER
• Responsible Party: PRINCIPAL INVESTIGATOR
• PI Title: Principal Investigator

Model Details: Participatory System Dynamics: Participatory system dynamics is a facilitated health care quality improvement or evidence-based practice implementation strategy that includes frontline addiction and mental health staff running simulations of clinic improvement strategies to find the best approaches for improving the reach of evidence-based psychotherapy and evidence-based pharmacotherapy. Audit and Feedback: Audit and feedback is a health care quality improvement or evidence-based practice implementation strategy that includes frontline addiction and mental health staff reviewing data to find the best approaches for improving the reach of evidence-based psychotherapy and evidence-based pharmacotherapy. Anticipate that 720 frontline providers will participate across both arms of this trial. There will be no interaction with current patients for the purposes of research. No new data will be collected beyond data generated during routine care.

Study Record Dates

• First Submitted: March 8, 2019
• First Posted: April 22, 2020
• Study Start: February 1, 2019
• Primary Completion (Estimated): July 31, 2026
• Study Completion (Estimated): January 29, 2027
• Last Updated: May 4, 2026

Record last verified: 2026-04

Data Sharing

• IPD Sharing: Will not share

Locations

US (1)