513 - Validity of a Novel Video Laryngoscope Competency Assessment Tool: A Simulation-Based Study

Poster Number: 513
Publication Number: 513.2721

Background: Competency in video laryngoscope (VL) intubation is paramount for effective management in paediatric airway emergencies. Currently there is no tool to assess competency for VL intubation skills. A VL competency assessment tool with supportive validity evidence is essential for evaluating the competency of healthcare professionals in VL intubation, ensuring high-quality patient outcomes.

Objective: To develop, evaluate and demonstrate validity evidence of a VL competency assessment tool using simulation.

Design/Methods: The assessment tool was crafted and reviewed by a core group of subject-matter experts, reflective of a previously published intubation assessment tool (Johnston, 2019), reviewed by a team of experts, integrating insights from existing assessment frameworks. The finalized tool comprised a 22-item checklist scaled from 0 to 2 (not done, correctly done or partially done/out of sequence), encapsulating various proficiency levels based on the Dreyfus model of skill acquisition (novice, advanced beginner, competent, proficient, expert) and a separate global skills assessment score. A selection of 10 simulated VL intubation videos were created, representing 2 distinct training levels. They were evaluated by paediatric emergency trained reviewers from three medical centres, following a comprehensive 3-hour online training session consisting of a thorough review and discussion of all individual items of the tool followed by individual performance ratings on 2 training videos. Post-training, the inter-rater reliability (IRR) was computed using intraclass correlation coefficients (ICC), and data analysis was conducted using SAS 9.4.

Results: The assessment tool exhibited robust validity, substantiated by substantial IRR scores following reviewer training. The ICC reflected substantial agreement among raters, with values of 0.837 (95%CI 0.612 – 0.970, p< 0.001) for preparation, 0.723 (95%CI: 0.511 – 0.897, p < 0.001) for intubation, and a cumulative score for absolute agreement of all items of 0.98 (95%CI 0.965 – 0.988) across all items, all proving statistically significant.

Conclusion(s): The study successfully birthed a validated VL competency assessment tool, fortified by solid IRR scores, thus paving the way for enhanced, simulation-supported, objective assessments of VL paediatric intubation skills. This tool holds significant promise as a cornerstone in future entrustable professional activities in the domain of VL intubation competency evaluation.