Comparison of Three Modalities to Assess Clinical Competence of Medical Students

NCT03884114 | Completed

• 1 other identifier: Ilumens0003
• Study Type: interventional
• Target: 42
• Locations: 1 country
• Sites: 1

Brief Summary

This study, focusing on the management of pediatric asthma exacerbations, assesses the clinical skills of medical students using three different evaluation tools: (i) the simulation game "Effic'Asthme" developed to train individuals on the management of pediatric asthma exacerbations; (ii) a multiple choice questionnaire (MCQ) on the same subject developed for the purpose of the study and (iii) high fidelity (HF)-simulation, considered as the gold-standard for its enhanced realism. Its objective is to determine which of the simulation game or the MCQ reflects the best the clinical competence of medical students evaluated on a HF simulator.

Conditions

Educational Problems

Keywords

technology-enhanced simulation; serious games; multiple choice questionnaires; medical education

Outcome Measures

Primary Outcomes (1)

• Items of the "asthma exacerbation performance score"

  This score uses a scale designed for the purpose of the study, the "Asthma exacerbation performance score", ranging from 0 to 19, 19 being the highest rate meaning that all the actions were correctly performed (1 point per action correctly performed). This score will allow to calculate the degree of correlation calculated by Kappa coefficient for each action performed or answer provided, between the simulation game and the high fidelity simulation session on the one hand; and between the multiple choice questionnaire and the high fidelity simulation session on the other hand; Then, we will compare the median Kappa coefficients for each condition, and determine which of the multiple choice questionnaire or the simulation game reflects the best the clinical competence of medical students observed during a high-fidelity simulation session.

  Baseline

Secondary Outcomes (2)

• The total score on the "asthma exacerbation performance score"

  Baseline

• The satisfaction of students using Likert-scales with each evaluation modality

  Baseline

Study Arms (1)

All participants

OTHER

All participants are evaluated on a scenario of pediatric asthma exacerbation using three different evaluation tools: (i) the simulation game EfficAsthme developed to train individuals on the management of pediatric asthma exacerbations; (ii) a MCQ on the same subject developed for the purpose of the study and (iii) HF-simulation, considered as the gold-standard for its enhanced realism.

Other: High-fidelity simulation; Other: Simulation game; Other: Multiple choice questionnaire

Interventions

High-fidelity simulation OTHER

The first evaluation modality studied is high fidelity (HF) simulation. In our study, this modality is considered to be the gold-standard assessment method of clinical assessment, because HF simulation corresponds to the modality which reflects the best clinical competence in hospital settings. The HF pediatric manikin used (SimBaby), is able to reproduce all the signs of an asthma exacerbation (coughing, wheezing, tachypnea, chest indrawing, seesaw respiration, cyanosis). In the simulation room, participants can use the same items that those present in the simulation game (a glass of water, paracetamol, short acting beta-agonist, controller treatment (Fluticasone), an asthma spacer with a facial mask, saline nose drops to perform nasal irrigation, oral steroids in tablets (prednisone), and a phone).

All participants

Simulation game OTHER

EfficAsthme is a simulation game used on a tablet computer. This simulation game was developed to train parents on the management of asthma exacerbations of their children. For the purpose of the study, EfficAsthme is diverted from its original use to assess students' clinical skills. The training scenario "A polluted atmosphere" is used in this study. Participants need to observe the signs presented by the child, and to determine the severity of the asthma exacerbation. From a menu on the right of the screen, the participant can choose several actions, especially to provide the short acting beta-agonist.

All participants

Multiple choice questionnaire OTHER

The third evaluation modality corresponds to a multiple choice questionnaire (MCQ) including 15 questions. As for HF-simulation and the simulation game, the MCQ starts with the same briefing and continues with 15 questions regarding the management of a moderate asthma exacerbation.

All participants

Eligibility Criteria

• Age: 18 Years+
• Sex: all
• Healthy Volunteers: Yes
• Age Groups: Adult (18-64), Older Adult (65+)

You may qualify if:

• medical students from Paris 5 University who meet the following criteria
• Being in their fifth year of medical school
• Having passed their pediatric exam in the previous 15 days
• Willing to participate in the study

You may not qualify if:

• refusal to participate

Sponsors & Collaborators

• Ilumens: lead

Study Sites (1)

Université Paris Descartes

Paris, 75270, France

Location

Related Publications (4)

• Norcini JJ, Swanson DB, Grosso LJ, Webster GD. Reliability, validity and efficiency of multiple choice question and patient management problem item formats in assessment of clinical competence. Med Educ. 1985 May;19(3):238-47. doi: 10.1111/j.1365-2923.1985.tb01314.x.

  PMID: 4010571 BACKGROUND

• Drummond D, Delval P, Abdenouri S, Truchot J, Ceccaldi PF, Plaisance P, Hadchouel A, Tesniere A. Serious game versus online course for pretraining medical students before a simulation-based mastery learning course on cardiopulmonary resuscitation: A randomised controlled study. Eur J Anaesthesiol. 2017 Dec;34(12):836-844. doi: 10.1097/EJA.0000000000000675.

  PMID: 28731928 BACKGROUND

• Adjedj J, Ducrocq G, Bouleti C, Reinhart L, Fabbro E, Elbez Y, Fischer Q, Tesniere A, Feldman L, Varenne O. Medical Student Evaluation With a Serious Game Compared to Multiple Choice Questions Assessment. JMIR Serious Games. 2017 May 16;5(2):e11. doi: 10.2196/games.7033.

  PMID: 28512082 BACKGROUND

• Fonteneau T, Billion E, Abdoul C, Le S, Hadchouel A, Drummond D. Simulation Game Versus Multiple Choice Questionnaire to Assess the Clinical Competence of Medical Students: Prospective Sequential Trial. J Med Internet Res. 2020 Dec 16;22(12):e23254. doi: 10.2196/23254.

  PMID: 33325833 DERIVED

Related Links

• Site showing the simulation game "EfficAsthme" developed

Study Officials

• David Drummond, MD, PhD

  Université Paris Descartes

  PRINCIPAL INVESTIGATOR

Study Design

• Study Type: interventional
• Phase: not applicable
• Masking: NONE
• Purpose: OTHER
• Intervention Model: SEQUENTIAL
• Sponsor Type: OTHER
• Responsible Party: SPONSOR

Model Details: All medical students are evaluated using a high-fidelity simulation environment, then using a serious game, then using multiple choice questionnaire, all on the same topic.

Study Record Dates

• First Submitted: March 18, 2019
• First Posted: March 21, 2019
• Study Start: March 20, 2019
• Primary Completion: July 6, 2019
• Study Completion: July 6, 2019
• Last Updated: July 20, 2020

Record last verified: 2020-07

Data Sharing

• IPD Sharing: Will share
• Shared Documents: STUDY PROTOCOL, SAP
• Time Frame: From baseline to 2 years after completion of the study.
• Access Criteria: Individual patient date, study protocol and statistical analysis plan will be available upon request to david.drummond@ilumens.org

Locations

FR (1)