306 - External Validation of Clinical Prediction Rules for Risk Stratification among Canadian Infants Presenting with Brief Resolved Unexplained Events (BRUEs)

Poster Number: 306
Publication Number: 306.185

Background: The American Academy of Pediatrics (AAP) guidelines classify BRUE infants into lower- and higher-risk. Studies after the guidelines’ introduction showed that while >90% of BRUE infants met the higher-risk criteria, only 5% had a serious condition. Furthermore, the AAP criteria showed low positive predictive value (PPV) and area under the curve (AUC) [0.52-0.54]. New prediction rules from a US cohort of 3,283 infants showed improved discrimination. Yet, these rules have not been validated in an external cohort.

Objective: To assess the prognostic accuracy of the AAP’s higher-risk criteria for serious underlying diagnosis and event recurrence and to externally validate the BRUE prediction rules.

Design/Methods: We analyzed data from 1,042 BRUE patients across 11 Canadian hospitals (2017-2021). Models’ performance was assessed using AUC for discrimination, and calibration plots. Predictors for a serious underlying diagnosis were multiple events, abnormal medical background, and age. Predictors of event recurrence were prematurity, multiple events, clusters, color change, respiratory irregularities, and tone changes.

We evaluated the original models and refined them using three methods, incrementally changing the original model. We first implemented Recalibration-in-the-Large, adjusting the original model’s intercept to the new dataset. This reflected differences in outcome prevalence between the American and Canadian datasets. Then, Logistic Recalibration was executed, modifying both intercept and slope. Lastly, we conducted a Model Revision, resulting in a logistic regression with new coefficients and intercept.

Results: Out of 1,042 patients, 977 (93.8%) were classified as higher-risk by the AAP. A total of 79 patients (7.6%) had a serious underlying diagnosis. For this outcome, the AAP criteria demonstrated a sensitivity of 100.0%, a PPV of 8.1%, and an AUC of 0.54 [95%CI: 0.53–0.54] (Table 1). The original prediction rules for identifying serious diagnoses displayed an AUC of 0.60 [0.54–0.67], which increased to 0.71 [0.65–0.76] after model revision (Figures 1 and 2). Event recurrence was noted in 163 patients (15.6%). For this outcome, the AAP criteria yielded a sensitivity of 99.4%, a PPV of 16.6%, and an AUC of 0.58 [0.56–0.58]. The prediction rule’s AUC was at 0.67 [0.62–0.72].

Conclusion(s): The clinical prediction rules outperformed the AAP criteria in external validation, and performance improved after recalibration. These rules provide clinicians and families with a tool for shared decision-making, facilitating personalized management grounded on individual risk tolerance.