Department of Medicine Connecticut Children's Medical Center Hartford, Connecticut, United States
Background: Patient and family violent outbursts towards staff, other family, or self-harm events have become more prevalent as the healthcare system experiences a behavioral health crisis. These events are likely under-reported. Objective: Assess the feasibility of using free-text provider notes to identify unreported events of healthcare-associated violence (HAV). Design/Methods: Our care settings included a tertiary care center and a community-based hospital. To construct a training set for the Natural Language Processing (NLP) model, we extracted all nursing handoff notes for all patients admitted to any inpatient unit in 2019. We followed a previously established workflow for data extraction using NLP, assisted by a manual review process, trained by two domain experts (an RN and an MD). The dashboard used to create the training set and later for ongoing surveillance is presented in Figure 1. We trained NLP models on tertiary care center data and validated the models on community hospital data. We first tested performance by applying the tertiary care models to the community hospital notes without modification, and then we conducted further evaluation after completing one round of local data categorization and re-training. We tested three transferred models: a support vector machine model, a binary logistic regression model using regular expressions (RegEx) as predictors, and a joint model where positive cases were those positive on either of the previously mentioned models. Finally, since the community hospital has a self-reported system for HAV events, we conducted real-time surveillance in 2022 to assess capture of unreported cases. Results: We extracted 70,981 notes from the tertiary care center and 19,332 notes from the community-based hospital. We calculated metrics for model performance and transferability (Table 1). Applying our surveillance workflow to the last 6 months of 2022, we identified 31 cases of HAV. Of these, 26 events (84%) involved staff and are considered reportable to existing patient safety systems per hospital regulation. During the same period, only 7 cases were self-reported to existing hospital systems, all of which were identified in our workflow. We categorized all notes by the nature of the incident (Figure 2).
Conclusion(s): NLP-assisted review is a feasible method for surveillance of HAVs within a hospital setting, with implementation and usability that can be achieved even at a low IT -resourced setting.
.png)
