CClinicalTrials.gg
CompletedNCT03884114SIMEVALUpdated Jul 20, 2020

Comparison of Three Modalities to Assess Clinical Competence of Medical Students

An interventional study of High-fidelity simulation and Simulation game in Educational Problems, sponsored by Ilumens. Completed at 1 site in France. Open to participants aged 18 Years and older, including healthy volunteers. Per ClinicalTrials.gov, last updated 2020-07-20.

Sponsored by Ilumens · Not applicable, Interventional, and Other

Phase
Not applicable
Study type
Interventional
Enrollment
42
Ages
18 Years and older
Sex
All
01

Study summary

This study, focusing on the management of pediatric asthma exacerbations, assesses the clinical skills of medical students using three different evaluation tools: (i) the simulation game "Effic'Asthme" developed to train individuals on the management of pediatric asthma exacerbations; (ii) a multiple choice questionnaire (MCQ) on the same subject developed for the purpose of the study and (iii) high fidelity (HF)-simulation, considered as the gold-standard for its enhanced realism.

Its objective is to determine which of the simulation game or the MCQ reflects the best the clinical competence of medical students evaluated on a HF simulator.

Read the detailed description

The assessment of medical students' clinical competence has long been of concern to educational institutions. Objective structural clinical examination (OSCE) and assessments in high-fidelity (HF) simulation settings represent interesting evaluation modalities but are associated with huge costs, especially when they are used for large-scale standardized assessments. Multiple choice questionnaires (MCQ) remain the most common evaluation tool in medical schools.

Simulation games may represent an interesting compromise between the cheap but limited assessment allowed by MCQs, and the comprehensive but highly expensive assessment allowed by OSCE and HF simulation.

This study, focusing on the management of pediatric asthma exacerbations, assesses the clinical skills of medical students using three different evaluation tools: (i) the simulation game "Effic'Asthme" developed to train individuals on the management of pediatric asthma exacerbations; (ii) a MCQ on the same subject developed for the purpose of the study and (iii) HF-simulation, considered as the gold-standard for its enhanced realism.

Its objective is to determine which of the simulation game or the MCQ reflects the best the clinical competence of medical students evaluated on a HF simulator.

02

Conditions studied

  • Educational Problems

Keywords

  • technology-enhanced simulation
  • serious games
  • multiple choice questionnaires
  • medical education
03

In context

Lead sponsor

Ilumens is the lead sponsor of 5 studies on the registry; none are open to participants now.

Counted across the registry records on this site, refreshed daily.

04

Who can participate

Ages eligible
18 Years and older
Sexes eligible
All
Accepts healthy volunteers
Yes

Inclusion criteria

  • medical students from Paris 5 University who meet the following criteria

    • Being in their fifth year of medical school
    • Having passed their pediatric exam in the previous 15 days
    • Willing to participate in the study

Exclusion criteria

Exclusion Criteria:

  • refusal to participate
05

Study design

Phase
Not applicable
Primary purpose
Other
Intervention model
Sequential assignment
Masking
None (open label)
Enrollment
42 participants (actual)

Study arms

  • Other
    All participants

    All participants are evaluated on a scenario of pediatric asthma exacerbation using three different evaluation tools: (i) the simulation game EfficAsthme developed to train individuals on the management of pediatric asthma exacerbations; (ii) a MCQ on the same subject developed for the purpose of the study and (iii) HF-simulation, considered as the gold-standard for its enhanced realism.

    Other: High-fidelity simulation · Other: Simulation game · Other: Multiple choice questionnaire

Interventions

  • OtherHigh-fidelity simulation

    The first evaluation modality studied is high fidelity (HF) simulation. In our study, this modality is considered to be the gold-standard assessment method of clinical assessment, because HF simulation corresponds to the modality which reflects the best clinical competence in hospital settings. The HF pediatric manikin used (SimBaby), is able to reproduce all the signs of an asthma exacerbation (coughing, wheezing, tachypnea, chest indrawing, seesaw respiration, cyanosis). In the simulation room, participants can use the same items that those present in the simulation game (a glass of water, paracetamol, short acting beta-agonist, controller treatment (Fluticasone), an asthma spacer with a facial mask, saline nose drops to perform nasal irrigation, oral steroids in tablets (prednisone), and a phone).

  • OtherSimulation game

    EfficAsthme is a simulation game used on a tablet computer. This simulation game was developed to train parents on the management of asthma exacerbations of their children. For the purpose of the study, EfficAsthme is diverted from its original use to assess students' clinical skills. The training scenario "A polluted atmosphere" is used in this study. Participants need to observe the signs presented by the child, and to determine the severity of the asthma exacerbation. From a menu on the right of the screen, the participant can choose several actions, especially to provide the short acting beta-agonist.

  • OtherMultiple choice questionnaire

    The third evaluation modality corresponds to a multiple choice questionnaire (MCQ) including 15 questions. As for HF-simulation and the simulation game, the MCQ starts with the same briefing and continues with 15 questions regarding the management of a moderate asthma exacerbation.

06

What researchers measure

Primary outcomes

  1. Items of the "asthma exacerbation performance score"

    This score uses a scale designed for the purpose of the study, the "Asthma exacerbation performance score", ranging from 0 to 19, 19 being the highest rate meaning that all the actions were correctly performed (1 point per action correctly performed). This score will allow to calculate the degree of correlation calculated by Kappa coefficient for each action performed or answer provided, between the simulation game and the high fidelity simulation session on the one hand; and between the multiple choice questionnaire and the high fidelity simulation session on the other hand; Then, we will compare the median Kappa coefficients for each condition, and determine which of the multiple choice questionnaire or the simulation game reflects the best the clinical competence of medical students observed during a high-fidelity simulation session.

    Time frame: Baseline

Secondary outcomes

  1. The total score on the "asthma exacerbation performance score"

    The total scores of this checklist will allow to calculate: * the degree of correlation of the scores using this checklist between the simulation game and the high-fidelity simulation session using spearman correlation, * the degree of correlation of the scores using this checklist between the simulation game and the multiple choice questionnaire using spearman correlation It will also allow to compare the median scores of the students between the three modalities of assessment (high fidelity simulation, simulation game and multiple choice questionnaire), and the dispersion of the scores between the three modalities.

    Time frame: Baseline

  2. The satisfaction of students using Likert-scales with each evaluation modality

    The satisfaction of students with the simulation game, and with the multiple choice questionnaire, will be assessed using a questionnaire with Likert-scales.

    Time frame: Baseline

07

Study locations

1 site
  • Université Paris Descartes
    Paris Cedex 06, 75270, France
08

References and documents

Publications

  • Norcini JJ, Swanson DB, Grosso LJ, Webster GD. Reliability, validity and efficiency of multiple choice question and patient management problem item formats in assessment of clinical competence. Med Educ. 1985 May;19(3):238-47. doi: 10.1111/j.1365-2923.1985.tb01314.x. PubMed 4010571 ↗
  • Drummond D, Delval P, Abdenouri S, Truchot J, Ceccaldi PF, Plaisance P, Hadchouel A, Tesniere A. Serious game versus online course for pretraining medical students before a simulation-based mastery learning course on cardiopulmonary resuscitation: A randomised controlled study. Eur J Anaesthesiol. 2017 Dec;34(12):836-844. doi: 10.1097/EJA.0000000000000675. PubMed 28731928 ↗
  • Adjedj J, Ducrocq G, Bouleti C, Reinhart L, Fabbro E, Elbez Y, Fischer Q, Tesniere A, Feldman L, Varenne O. Medical Student Evaluation With a Serious Game Compared to Multiple Choice Questions Assessment. JMIR Serious Games. 2017 May 16;5(2):e11. doi: 10.2196/games.7033. PubMed 28512082 ↗
  • Fonteneau T, Billion E, Abdoul C, Le S, Hadchouel A, Drummond D. Simulation Game Versus Multiple Choice Questionnaire to Assess the Clinical Competence of Medical Students: Prospective Sequential Trial. J Med Internet Res. 2020 Dec 16;22(12):e23254. doi: 10.2196/23254. PubMed 33325833 ↗

Individual participant data

Plan to share: Yes — * Participants' characteristics anonymized * Participants' correct and incorrect actions and answers when evaluated on the 3 different modalities: multiple choice questionnaire, high-fidelity simulation and simulation game

Supporting information: Study protocol, Sap

09

Updates

Tracking since Sep 25, 2026
No changes since tracking began. The registry record was last updated on Jul 20, 2020, before this site started recording changes on Sep 25, 2026. Its history is on ClinicalTrials.gov ↗
10

Registry details

Key details

Study ID
NCT03884114
Lead sponsor
Ilumens
Responsible party
Sponsor
First posted
Mar 21, 2019
Start date
Mar 20, 2019
Primary completion
Jul 6, 2019
Completion
Jul 6, 2019
Last update
Jul 20, 2020

Study contacts

David Drummond, MD, PhD
principal investigator · Université Paris Descartes

Oversight

Data monitoring committee
No
FDA-regulated drug
No
FDA-regulated device
No
View the source record on ClinicalTrials.gov ↗

Not currently enrolling

This study is completed, as verified in Jul 2020. You cannot join it, but the record below documents what was studied.

Follow this study

Get an email when the registry record changes — status, dates, results — or when someone posts here.

Sign in to follow

Discussion

Questions and observations about this study, from anyone following it. Not medical advice, and not a channel to the study team — their contact details are on the registry record.

Sign in to join the discussion. Reading takes no account; posting does. You choose a display name, and a pseudonym is the default.

Nothing here yet. If you are running this trial, taking part in it, or weighing whether to, this is the place to say so.

Start the discussion