The following is a summary of “Evaluation of a Model to Target High-risk Psychiatric Inpatients for an Intensive Postdischarge Suicide Prevention Intervention,” published in the January 2023 issue of Psychiatry by Kessler, et al.

The period following discharge from a psychiatric hospital presents a high risk of suicide, and while intensive case management after discharge can be effective in preventing suicide, it is only cost-effective if targeted at high-risk patients. A previously developed machine learning (ML) model showed that postdischarge suicides could be predicted from electronic health records and geospatial data, but it was unclear if prediction could be improved with additional information.

For a study, researchers sought to determine if model prediction could be improved by adding information extracted from clinical notes and public records. The models were trained to predict suicides in the 12 months after short-term (less than 365 days) psychiatric hospitalizations in the Veterans Health Administration (VHA) [(299,050 hospitalizations)] between the beginning of 2010 and September 1, 2012, and tested in hospitalizations from September 2, 2012, to December 31, 2013 [(149,738 hospitalizations, with 393 hospitalizations followed within 12 months by suicides)]. The analysis focused on net benefit across a range of plausible decision thresholds, and predictor importance was assessed using Shapley additive explanations (SHAP) values. The data were analyzed from January to August 2022.

The main outcomes and measures of the study were suicides, defined by the National Death Index. The base model predictors included VHA electronic health records and patient residential data, while the expanded predictors were obtained from natural language processing (NLP) of clinical notes and the social determinants of health (SDOH) public records database.

The model included 448,788 unique hospitalizations, and the net benefit over risk horizons between 3 and 12 months was generally highest for the model that included both NLP and SDOH predictors [(area under the receiver operating characteristic curve range, 0.747-0.780; area under the precision-recall curve relative to the suicide rate range, 3.87-5.75)]. NLP and SDOH predictors also had the highest predictor class-level SHAP values [(proportional SHAP = 64.0% and 49.3%, respectively)], although the single highest positive variable-level SHAP value was for a count of medications classified by the US Food and Drug Administration as increased suicide risk prescribed the year before hospitalization [(proportional SHAP = 15.0%)].

The study concluded that clinical notes and public records could improve ML model prediction of suicide after psychiatric hospitalization, and the model had a positive net benefit over 3-month to 12-month risk horizons for plausible decision thresholds. However, the study cautioned against inferring causality based on predictor importance and recommends investigating key predictors’ potential intervention implications in future studies.

Reference: jamanetwork.com/journals/jamapsychiatry/article-abstract/2800171