612 - Inter-Rater Reliability of Parent and Teacher Reported Vanderbilt Assessment Scales Before and During Remote Learning in Screening for ADHD in an Urban Multiethnic Developmental-Behavioral Center

Poster Number: 612
Publication Number: 612.2084

Background: Attention deficit hyperactivity disorder (ADHD) is common in children. The Vanderbilt Assessment Scale supports diagnosis of ADHD per DSM-5 with parents (VADPRS) and teachers (VADTRS) as informants. During COVID-19 pandemic, students in NYC were mandated to attend school remotely. There are currently no studies to determine the degree of agreement of the VADPRS and VADTRS during in-person and remote learning.

Objective: To compare inter-rater reliability, sensitivity, specificity, positive (PPV) and negative predictive value (NPV) of VADPRS and VADTRS during in-person vs. remote learning.

Design/Methods: Retrospective study of children aged 6 to 12 years referred to the Developmental and Behavioral Center for ADHD evaluation from 2019 to 2022. Subjects evaluated from Jan 1, 2019 to Mar 1, 2020 as in-person classroom (G1) were compared to subjects seen from Apr 1, 2020 to Aug 30, 2021 during remote learning (G2). We excluded patients with autism and those without the Vanderbilt on file. A stratified random selection procedure identified 60 patients for each group for chart review. Kappa (κ) coefficients ranging from -1 to +1 were used to determine level of agreement, p< 0.05 was considered significant.

Results: Of 270 patients that met the inclusion criteria, 86 (32%) were in G1 and 184 (68%) in G2. The majority were Hispanic (61%) and male (73%) with an average age of 8.3±1.8 years. The groups did not differ on rates of confirmed ADHD diagnosis (G1=41, G2=42). Agreement overall was fair for the full sample (κ=0.33, p< 0.001). During in-person learning, agreement was moderate (κ=0.48, p< 0.001) but was none to slight during remote learning (κ=0.19, p=0.14). Parent reports (G1 vs G2) showed improved sensitivity (0.68 vs 0.72), specificity (0.73 vs 0.79), PPV (0.54 vs 0.59) and NPV (0.83 vs 0.87) during remote learning. Teacher reports (G1 vs G2) had increased sensitivity (0.74 vs 0.78) and NPV (0.86 vs 0.88) while specificity (0.78 vs 0.67) and PPV (0.61 vs 0.50) decreased during remote learning. Confirmed ADHD diagnoses were highest when parents and teachers agreed on the Vanderbilt assessment showing the strongest sensitivity (0.89 vs 0.94) and NPV (0.93 vs 0.96), and good specificity (0.66 vs 0.52) and PPV (0.54 vs 0.46).

Conclusion(s): VADPRS and VADTRS exhibited moderate level of agreement during in-person learning and none to slight level of agreement during remote learning. The scales showed the strongest predictive value with teacher-parent symptom agreement, minimizing false positives. VADTRS used in isolation in remote learning were associated with greater false negatives.