Do Script Concordance Tests Correlate With Family Medicine Standardized Tests and Failing Rotation Grades?

NCT03938610 | Active Not Recruiting

• 1 other identifier: FWH20190010H
• Study Type: observational
• Target: 79
• Locations: 1 country
• Sites: 1

Brief Summary

Delayed or poor identification and remediation of clinical reasoning difficulties can lead to clinician underperformance and can ultimately compromise patient care. To date, no research has been done to see if SCTs correlate with current grading standards (ITE, ABFM certification exam score, ACGME milestones, or failing clinical rotation grades) in Family Medicine Residents. If investigators can identify that an SCT correlates with current standardized testing of Family medicine residents, it could be possible to identify struggling learners prior to poor scores on the ITE, ACGME milestones, or clinical rotations. If a learner does poorly on an SCT early in the academic year the learner can begin a remediation plan to improve their deficits before receiving a failing grade, poor ITE or ACGME milestone scores.

Conditions

Knowledge, Attitudes, Practice

Outcome Measures

Primary Outcomes (24)

• SCT scores compared to ITE scores

  SCT scores, ranging from 0 to 100 percent, are longitudinal and will be recorded for all PGY1, 2, and 3 subjects enrolled in the study each year of their residency in June or July.

  year 1

• SCT scores compared to ITE scores

  SCT scores, ranging from 0 to 100 percent, are longitudinal and will be recorded for all PGY1, 2, and 3 subjects enrolled in the study each year of their residency in June or July.

  year 2

• SCT scores compared to ITE scores

  SCT scores, ranging from 0 to 100 percent, are longitudinal and will be recorded for all PGY1, 2, and 3 subjects enrolled in the study each year of their residency in June or July.

  year 3

• SCT scores compared to ABFM scores

  SCT scores, ranging from 0 to 100 percent, are longitudinal and will be recorded for all PGY1, 2, and 3 subjects enrolled in the study each year of their residency in June or July.

  year 1

• SCT scores compared to ABFM scores

  SCT scores, ranging from 0 to 100 percent, are longitudinal and will be recorded for all PGY1, 2, and 3 subjects enrolled in the study each year of their residency in June or July.

  year 2

• SCT scores compared to ABFM scores

  SCT scores, ranging from 0 to 100 percent, are longitudinal and will be recorded for all PGY1, 2, and 3 subjects enrolled in the study each year of their residency in June or July.

  year 3

• SCT scores compared to family medicine rotation grades

  SCT scores, ranging from 0 to 100 percent, are longitudinal and will be recorded for all PGY1, 2, and 3 subjects enrolled in the study each year of their residency in June or July.

  year 1

• SCT scores compared to family medicine rotation grades

  SCT scores, ranging from 0 to 100 percent, are longitudinal and will be recorded for all PGY1, 2, and 3 subjects enrolled in the study each year of their residency in June or July.

  year 2

• SCT scores compared to family medicine rotation grades

  SCT scores, ranging from 0 to 100 percent, are longitudinal and will be recorded for all PGY1, 2, and 3 subjects enrolled in the study each year of their residency in June or July.

  year 3

• SCT scores compared to ACGME milestones.

  SCT scores, ranging from 0 to 100 percent, are longitudinal and will be recorded for all PGY1, 2, and 3 subjects enrolled in the study each year of their residency in June or July.

  year 1

• SCT scores compared to ACGME milestones.

  SCT scores, ranging from 0 to 100 percent, are longitudinal and will be recorded for all PGY1, 2, and 3 subjects enrolled in the study each year of their residency in June or July.

  year 2

• SCT scores compared to ACGME milestones.

  SCT scores, ranging from 0 to 100 percent, are longitudinal and will be recorded for all PGY1, 2, and 3 subjects enrolled in the study each year of their residency in June or July.

  year 3

• Binomial SCT classification derived from each interval SCT scores compared to ACGME milestones.

  A "fail" classification is defined as an SCT score 1 SD below the respective PGY mean. All other SCT scores will be classified as "pass".

  year 1

• Binomial SCT classification derived from each interval SCT scores compared to ACGME milestones.

  A "fail" classification is defined as an SCT score 1 SD below the respective PGY mean. All other SCT scores will be classified as "pass".

  year 2

• Binomial SCT classification derived from each interval SCT scores compared to ACGME milestones.

  A "fail" classification is defined as an SCT score 1 SD below the respective PGY mean. All other SCT scores will be classified as "pass".

  year 3

• Binomial SCT classification derived from each interval SCT scores compared to family medicine rotation grades.

  A "fail" classification is defined as an SCT score 1 SD below the respective PGY mean. All other SCT scores will be classified as "pass".

  year 1

• Binomial SCT classification derived from each interval SCT scores compared to family medicine rotation grades.

  A "fail" classification is defined as an SCT score 1 SD below the respective PGY mean. All other SCT scores will be classified as "pass".

  year 2

• Binomial SCT classification derived from each interval SCT scores compared to family medicine rotation grades.

  A "fail" classification is defined as an SCT score 1 SD below the respective PGY mean. All other SCT scores will be classified as "pass".

  year 3

• Binomial SCT classification derived from each interval SCT scores compared to ITE

  A "fail" classification is defined as an SCT score 1 SD below the respective PGY mean. All other SCT scores will be classified as "pass".

  year 1

• Binomial SCT classification derived from each interval SCT scores compared to ITE

  A "fail" classification is defined as an SCT score 1 SD below the respective PGY mean. All other SCT scores will be classified as "pass".

  year 2

• Binomial SCT classification derived from each interval SCT scores compared to ITE

  A "fail" classification is defined as an SCT score 1 SD below the respective PGY mean. All other SCT scores will be classified as "pass".

  year 3

• Binomial SCT classification derived from each interval SCT scores compared to ABFM

  A "fail" classification is defined as an SCT score 1 SD below the respective PGY mean. All other SCT scores will be classified as "pass".

  year 1

• Binomial SCT classification derived from each interval SCT scores compared to ABFM

  A "fail" classification is defined as an SCT score 1 SD below the respective PGY mean. All other SCT scores will be classified as "pass".

  year 2

• Binomial SCT classification derived from each interval SCT scores compared to ABFM

  A "fail" classification is defined as an SCT score 1 SD below the respective PGY mean. All other SCT scores will be classified as "pass".

  year 3

Eligibility Criteria

• Sex: all
• Healthy Volunteers: No
• Age Groups: Child (0-17), Adult (18-64), Older Adult (65+)
• Sampling Method: Non-Probability Sample

Study Population

Family medicine residents at 99MDG

You may qualify if:

• Family Medicine resident at 99MDG

You may not qualify if:

• \-- N/A

Sponsors & Collaborators

• Pamela Hughes: lead

Study Sites (1)

Mike O'Callaghan Military Medical Center

Las Vegas, Nevada, 89191, United States

Location

Related Publications (3)

• Gagnon R, Charlin B, Coletti M, Sauve E, van der Vleuten C. Assessment in the context of uncertainty: how many members are needed on the panel of reference of a script concordance test? Med Educ. 2005 Mar;39(3):284-91. doi: 10.1111/j.1365-2929.2005.02092.x.

  PMID: 15733164 BACKGROUND

• Fournier JP, Demeester A, Charlin B. Script concordance tests: guidelines for construction. BMC Med Inform Decis Mak. 2008 May 6;8:18. doi: 10.1186/1472-6947-8-18.

  PMID: 18460199 BACKGROUND

• O'Neill TR, Li Z, Peabody MR, Lybarger M, Royal K, Puffer JC. The Predictive Validity of the ABFM's In-Training Examination. Fam Med. 2015 May;47(5):349-56.

  PMID: 25905876 BACKGROUND

MeSH Terms

Conditions

Behavior

Study Officials

• Paul F Crawford, MD

  US Air Force

  STUDY DIRECTOR

Study Design

• Study Type: observational
• Observational Model: COHORT
• Time Perspective: PROSPECTIVE
• Sponsor Type: FED
• Responsible Party: SPONSOR INVESTIGATOR
• PI Title: Principal Investigator

Study Record Dates

• First Submitted: May 1, 2019
• First Posted: May 6, 2019
• Study Start: June 1, 2019
• Primary Completion (Estimated): June 1, 2026
• Study Completion (Estimated): June 1, 2026
• Last Updated: July 23, 2025

Record last verified: 2025-07

Data Sharing

• IPD Sharing: Will not share

Locations

US (1)