Evaluation of a Remote Training Strategy

NCT05039164 | Completed DMC

• 1 other identifier: 21-018863
• Study Type: interventional
• Target: 140
• Locations: 1 country
• Sites: 1

Brief Summary

An increasing number of schools in rural settings are employing the multi-tier positive behavioral interventions and supports (PBIS) framework to improve school-climate. PBIS can also be used as a framework for the adoption and integration of evidence-based practices (EBPs) for children's mental health concerns. A key challenge is that school personnel need technical assistance (training plus ongoing consultation) in order to implement EBPs with fidelity. In urban and suburban schools, this support can be provided to school staff on site. However, providing ongoing on-site support is not feasible or sustainable in the majority of rural schools, due to their remote physical location. For this reason, video technology has been recommended for the training of behavioral health staff (BHS) in under-served rural communities.

Conditions

Behavior Disorders

Keywords

Positive-behavior Interventions and Supports; Evidence-based Practices; Tier 2; Mental Health; Rural Schools

Outcome Measures

Primary Outcomes (7)

• Measures of content fidelity for group cognitive-behavioral treatment (GCBT)

  The primary endpoints related to school staff implementing the interventions are measures of content fidelity. Content fidelity for the group interventions CPP and CATS will be measured using the Content Fidelity Checklists that reflects each activity component of the session agenda of the treatment protocols. Raters use a yes/no response scale to indicate whether or not the implementer covered a particular component.

  Up to 3 years

• Measures of content fidelity for Check-in, Check-Out

  Content fidelity for CI/CO will be assessed via the Check-In/Check-Out Fidelity Checklist. The CI/CO Fidelity Checklist is completed by the consultant (research staff) on a monthly basis after collecting the completed Daily Progress Report (DPR) student forms for each student involved in CICO.

  Up to 3 years

• Measures of process fidelity

  The primary endpoints related to school staff implementing the interventions are measures of content and process fidelity. Process fidelity for the group interventions CPP and CATS will be assessed via an adapted version of the rating system developed by Lochman and colleagues. 70 Ten items are rated on a scale of 0 to 4, with 0 being "Not At All" and 4 being "Very Often". Ratings are given on the extent to which school staff members delivered the intervention in an orderly fashion, using active learning strategies. etc. Exploratory factor analysis (EFA) yielded two factors on the 10-item scale.

  Up to 3 years

• Perceived adoption and penetration

  Perceived adoption and penetration will be assessed via a survey completed by school staff. The adoption inventory will track the number of times each intervention is used per school, per condition. The penetration inventory will list EBP penetration at the student level (students receiving EBPs at Tier 2).

  Up to 3 years

• Changes in student mental health symptoms from Baseline to Post Intervention

  Student Mental Health Symptoms will be measured by comparing parent scores pre-intervention and post-intervention. The Behavior Assessment System for Children-3rd edition (BASC-3) is a 138-item, 4-point, Likert-type rating scale (N=Never, S=Sometimes, O=Often, A=Almost Always) for assessing parental report of child mental health functioning, standardized for ages 2.5 to 18 years.

  Baseline and 14 weeks (post intervention)

• Changes in academic engagement from Baseline to Post Intervention

  Student Academic Engagement will be measured by comparing teacher scores pre-intervention and post-intervention. This is a 20-item, four-point instrument (1 = Not At All True, 2 = Not Very True, 3 = Sort Of True, 4 = Very True) with four sub-scales. Investigators will use the average score for each of the four scales at pre- and post-participation in group cognitive behavioral therapy (CBT) or Check-in/Check-out (CICO).

  Baseline and 14 weeks (post intervention)

• Changes in externalizing and internalizing problems

  Student externalizing and internalizing problems will be measure by comparing student scores pre-intervention and post-intervention. The Behavior and Feeling Survey-Youth Report (YFS-SR) is a brief (10 items) measure of internalizing and externalizing problems rated on a 5-point scale. The YFS-SR has excellent psychometric properties, and it is sensitive to change. The YFS-SR provides three scores (Internalizing, Externalizing, Total).

  Baseline and 14 weeks (post intervention)

Secondary Outcomes (1)

• Perceived mediators and moderators of consultation support

  Up to 3 years

Study Arms (3)

Initial training

ACTIVE COMPARATOR

School personnel will implement CATS or CPP and CICO with students in the school setting. They will participate in an initial live remote training to learn about implementing the three EBPs (CC).

Behavioral: Coping Power Program (CPP); Behavioral: CBT for Anxiety Treatment in Schools (CATS); Behavioral: Check-in/Check-out (CI/CO); Behavioral: Initial Training (CC)

Initial training plus video

EXPERIMENTAL

School personnel will implement CATS or CPP and CICO with students in the school setting. They will participate in an initial live remote training and receive access to asynchronous video training modules about the EBPs (RV).

Behavioral: Coping Power Program (CPP); Behavioral: CBT for Anxiety Treatment in Schools (CATS); Behavioral: Check-in/Check-out (CI/CO); Behavioral: Initial Training plus Video (RV)

Initial training plus video, plus coaching

EXPERIMENTAL

School personnel will implement CATS or CPP and CICO with students in the school setting. They will participate in an initial live remote training, receive access to asynchronous video training modules about the EBPs, and receive coaching support by study staff (RV+). The coaching will be from a study consultant regarding the implementation of EBPs.

Behavioral: Coping Power Program (CPP); Behavioral: CBT for Anxiety Treatment in Schools (CATS); Behavioral: Check-in/Check-out (CI/CO); Behavioral: Initial Training plus Video plus Coaching (RV+)

Interventions

Coping Power Program (CPP) BEHAVIORAL

CPP is an evidence-based intervention designed for students with externalizing behavior disorder. CPP consists of twelve 45-minute sessions. This EBP has been found to be effective at reducing aggressive behavior, covert delinquent behavior and substance abuse among aggressive boys, with gains maintained at one-year follow-up. Growth curve analyses showed that CPP had linear effects for three years after intervention on reductions in aggressive behavior and academic behavior problems.

Initial training; Initial training plus video; Initial training plus video, plus coaching

CBT for Anxiety Treatment in Schools (CATS) BEHAVIORAL

CATS is an adaptation of Friends for Life (FRIENDS). The adapted protocol retains the core elements of evidence-based CBT for anxiety and the FRIENDS group format. Investigators implemented planned adaptations to the protocol based on collective experience. Changes were made to the language, cultural methods, number of sessions, and activities while maintaining the 5 essential components of the treatment. This resulted in a briefer (8-session) and more feasible, engaging and culturally appropriate protocol for urban under-resourced schools than the original FRIENDS.

Initial training; Initial training plus video; Initial training plus video, plus coaching

Check-in/Check-out (CI/CO) BEHAVIORAL

CI/CO is a targeted, Tier 2 intervention for students at risk of developing externalizing and internalizing mental health disorders. CI/CO is designed to provide immediate feedback (i.e., at the end of each class period) to students, based on the use of a daily report card. This feedback is developmentally sensitive. CI/CO implementers meet individually with students for a brief 'check-in' in the morning and a brief 'check-out' in the afternoon. Research on the use of CI/CO has shown it to be effective in reducing externalizing and internalizing problems with elementary school students. CI/CO will be offered to individual students for a three-month period of time, which is the same time-frame needed for the implementation of CPP.

Initial training; Initial training plus video; Initial training plus video, plus coaching

Initial Training (CC) BEHAVIORAL

School personnel will participate in an initial live remote training.

Initial training

Initial Training plus Video (RV) BEHAVIORAL

School personnel will participate in an initial live remote training and receive access to asynchronous video training modules.

Initial training plus video

Initial Training plus Video plus Coaching (RV+) BEHAVIORAL

School personnel will participate in an initial live remote training, receive access to asynchronous video training modules, and receive coaching support from study staff.

Initial training plus video, plus coaching

Eligibility Criteria

• Age: 8 Years - 99 Years
• Sex: all
• Healthy Volunteers: Yes
• Age Groups: Child (0-17), Adult (18-64), Older Adult (65+)

You may qualify if:

• Administrator: Any school principal or assistant principal from participating schools implementing PBIS.
• Behavioral Health Staff: Any counselor, social worker, or teacher from participating schools implementing PBIS who work with students in grades 4-8.
• attending one of the participating schools
• being in grades 4-8
• identified by the Tier 2 team as not responding to Tier 1 intervention, thus needing Tier 2 support
• scoring ≥ 1 SD above the mean on the Emotional Symptoms or Conduct Problems scales of the Strength andDifficulties Questionnaire (SDQ) completed by a parent or a teacher
• The cut-off score level for the SDQ is appropriate for identifying students at risk for a behavioral/mental health disorder. We limit participation to students in grades 4-8 because the group EBPs are appropriate for this age group.

You may not qualify if:

• Administrator: School staff who are not principals or assistant principals.
• Behavioral Health Staff: School staff who are not part of the PBIS team and who do not work with students in grades 4-8.
• Students: Students who do not meet screening or group participation criteria will not be included in the study. Students with a history of intellectual disability or serious developmental delays according to school records will not be included because they would be unlikely to benefit from the interventions used in the study. Students with a history of psychotic or autistic spectrum disorders as reported by parents will not be included.
• Subjects that do not meet all the enrollment criteria may not be enrolled. Any violations of these criteria must be reported in accordance with the Institutional Review Board (IRB) Policies and Procedures.

Sponsors & Collaborators

• Children's Hospital of Philadelphia: lead
• Devereux Center for Effective Schools: collaborator
• Agency for Healthcare Research and Quality (AHRQ): collaborator

Study Sites (1)

Children's Hospital of Philadelphia

Philadelphia, Pennsylvania, 19104, United States

Location

Related Publications (1)

• Eiraldi R, McCurdy BL, Khanna MS, Goldstein J, Comly R, Francisco J, Rutherford LE, Wilson T, Henson K, Farmer T, Jawad AF. Development and evaluation of a remote training strategy for the implementation of mental health evidence-based practices in rural schools: pilot study protocol. Pilot Feasibility Stud. 2022 Jun 17;8(1):128. doi: 10.1186/s40814-022-01082-4.

  PMID: 35710520 DERIVED

MeSH Terms

Conditions

Mental Disorders; Psychological Well-Being

Interventions

Schools; Videotape Recording

Condition Hierarchy (Ancestors)

Personal Satisfaction; Behavior

Intervention Hierarchy (Ancestors)

Non-Medical Public and Private Facilities; Tape Recording; Audiovisual Aids; Educational Technology; Technology; Technology, Industry, and Agriculture; Television

Study Officials

• Ricardo B Eiraldi, PhD

  Children's Hospital of Philadelphia

  PRINCIPAL INVESTIGATOR

Study Design

• Study Type: interventional
• Phase: not applicable
• Allocation: RANDOMIZED
• Masking: NONE
• Purpose: HEALTH SERVICES RESEARCH
• Intervention Model: PARALLEL
• Sponsor Type: OTHER
• Responsible Party: SPONSOR

Study Record Dates

• First Submitted: August 26, 2021
• First Posted: September 9, 2021
• Study Start: September 20, 2022
• Primary Completion: June 13, 2025
• Study Completion: June 13, 2025
• Last Updated: August 15, 2025

Record last verified: 2025-08

Locations

US (1)