A Study to Evaluate Strategies for Teaching Effective Use of Diagnostic Tests

NCT04130607 | Completed

• 1 other identifier: 18-04-EX-0062
• Study Type: interventional
• Target: 65
• Locations: 1 country
• Sites: 1

Brief Summary

A recent Institute of Medicine monograph brought attention to high rates of diagnostic error and called for better educational efforts to improve diagnostic accuracy.1 Educational methods, however, are rarely tested and some educational efforts may be ineffective and wasteful.2 In this study, we plan to examine whether explicit instruction on diagnostic methods will have an effect on diagnostic accuracy of 2nd-year medical students and internal medicine residents.

Conditions

Instructional Methods

Keywords

Teaching Strategies, Diagnostic reasoning

Outcome Measures

Primary Outcomes (1)

• Accuracy of participants probability revisions were compared to posttest probability revisions that were calculated using Bayes Rule. An effect size was calculated to measure how close students matched the calculated revision.

  To perform the effect size analysis, two transformations were performed. First, the difference between the subjective estimate and the Bayesian calculation of post-test probability was squared to remove negative differences and permit combining of the effects of positive and negative test results. Second, a correction based on the intrinsic error of a probability estimate was applied by dividing each squared difference by p(1-p). In this manner, we transformed each raw difference to a squared effect size (difference / error of difference). Finally, the square root was computed, to transform the data back to an effect size. The resulting effect size was then used for statistical analysis. For this primary analysis, a mixed model ANOVA was used.

  Post-test was taken within 72 hours of instructional phase completion.

Study Arms (3)

Analytical

EXPERIMENTAL

Students will receive brief instruction in probability, sensitivity, specificity, and likelihood ratios, with distributions and calculations. Pretest and posttest probabilities will be computed for two cases for each of the three conditions listed above.

Other: Conceptual teaching

Experiential

ACTIVE COMPARATOR

Students will receive a brief instruction conceptually discussing sensitivity and specificity (e.g. "a sensitive test will be positive at even low levels of disease. However, this can lead to a number of false positive errors, when the test is positive even when there is no disease. As a result, it is most useful for ruling out a diagnosis"). They will then work through a total of 30 cases, 10 for each condition, in blocked sequence. For each brief written case they will be asked for a probability of diagnosis after the clinical information is presented. The test result will then be given and they will be asked for a post-test probability. Their estimate will be compared to the computed value based on published estimates of sensitivity and specificity and feedback provided.

Other: Conceptual teaching

No Explicit Instruction or Examples

PLACEBO COMPARATOR

Students will receive 3 passages from a clinical text related to each of the 3 conditions in the study and asked to study them for 15 min each.

Other: Conceptual teaching

Interventions

Conceptual teaching OTHER

The present study is designed to contrast two instructional methods - explicit instruction in likelihood ratios and pretest/posttest probabilities versus implicit instruction based on presentation of multiple cases. These will be compared to a "no intervention" control group.

Also known as: Teaching through examples, No active teaching

Analytical; Experiential; No Explicit Instruction or Examples

Eligibility Criteria

• Age: 18 Years+
• Sex: all
• Healthy Volunteers: Yes
• Age Groups: Adult (18-64), Older Adult (65+)

You may qualify if:

• Medical Student at McMaster University or Eastern Virginia Medical School
• Completed 18 months of coursework

Sponsors & Collaborators

• Sentara Norfolk General Hospital: lead
• McMaster University: collaborator

Study Sites (1)

Sentara Norfolk General Hospital

Norfolk, Virginia, 23507, United States

Location

Related Publications (1)

• Brush JE Jr, Lee M, Sherbino J, Taylor-Fishwick JC, Norman G. Effect of Teaching Bayesian Methods Using Learning by Concept vs Learning by Example on Medical Students' Ability to Estimate Probability of a Diagnosis: A Randomized Clinical Trial. JAMA Netw Open. 2019 Dec 2;2(12):e1918023. doi: 10.1001/jamanetworkopen.2019.18023.

  PMID: 31860107 DERIVED

Study Design

• Study Type: interventional
• Phase: not applicable
• Allocation: RANDOMIZED
• Masking: SINGLE
• Who Masked: PARTICIPANT
• Masking Details: Participants are not aware of the other arms in the trial
• Purpose: HEALTH SERVICES RESEARCH
• Intervention Model: PARALLEL
• Sponsor Type: OTHER
• Responsible Party: PRINCIPAL INVESTIGATOR
• PI Title: Cardiologist

Model Details: Randomized trial of teaching methods with 3 arms.

Study Record Dates

• First Submitted: October 15, 2019
• First Posted: October 17, 2019
• Study Start: May 15, 2018
• Primary Completion: January 1, 2019
• Study Completion: October 15, 2019
• Last Updated: October 17, 2019

Record last verified: 2019-10

Data Sharing

• IPD Sharing: Will not share

Locations

US (1)