A recent study shows that diagnosis codes miss most self-harm cases recorded in veterans’ medical files, pointing to a gap in how mental health is tracked and managed.
Crucial mental health details are often hidden deep within medical records, making them hard for clinicians to find and use. This is especially true for self-harm, which doesn’t always show up in the diagnosis codes that hospitals and clinics depend on. A new study from the University of New Mexico School of Medicine highlights just how much can be overlooked when only diagnosis codes are used.
What lies beneath the surface of patient records
Medical records hold a wealth of information about a patient’s mental health, but not all of it is easy to access. Diagnosis codes, widely used by clinicians and researchers to identify and track conditions, often miss key facts. In this study, researchers examined electronic health records from over 1.3 million Veterans Health Administration patients. They found that diagnosis codes picked up only about one in four self-harm cases that clinicians had documented in the notes.
This means that many self-harm cases remain hidden when health systems rely only on diagnosis codes. Because of this, hospitals and clinics may not see the full need for mental health care, and researchers could underestimate how widespread the problem is. This lack of visibility can affect everything from planning care to deciding how to allocate resources for mental health.
Even summary sections like problem lists, which are meant to highlight important health issues for care teams, often left out this history. For veterans who had a diagnosis code for self-harm, fewer than a quarter had it listed in their problem list. This shows that even when self-harm is recognized, it is often not recorded where care teams are most likely to see it.
The hidden numbers paint a different picture
The study, published in the Journal of Medical Internet Research, used a new machine learning method to dig deeper into the data. After reviewing charts and using advanced statistical tools, the researchers estimated that 7.9 percent of veterans seen by VHA clinicians had a documented history of self-harm. In contrast, only 1.85 percent of cases were visible through diagnosis codes.
This means diagnosis codes alone miss more than three-quarters of self-harm cases. This gap matters because a history of self-harm is a strong warning sign for future risk, including suicide. Missing this information can mean missed chances to step in with support and care.
Christophe Lambert, PhD, the corresponding author of the study, explained, “For research and planning, if we only count what is easy to see in diagnosis codes, we may substantially underestimate the need for mental health services. Better measurement can help health systems plan better, help researchers study care more accurately and eventually help clinicians know when a patient may need a closer look.”
Why these findings are changing the conversation
Knowing if a patient has a history of self-harm is vital for clinicians. It helps them assess risk and plan the best care for issues like depression, PTSD, and substance use. When this history is missing from easy-to-find parts of the record, important warning signs can be overlooked.
Christophe Lambert, PhD, noted, “This is a systems-level visibility problem. The record can be enormous. In our chart review, some patient records had more than 500,000 lines of notes. No clinician can be expected to read all of that during a normal visit.”
The study points to a bigger systems problem: medical records can be massive and hard to review in detail.
New technology uncovers what’s been missed
The research team used a method called PULSNAR (Positive Unlabeled Learning Selected Not At Random), designed for real-world medical data. Most machine learning tools need clear examples of both “yes” and “no” cases. But in medical records, the absence of a diagnosis code doesn’t mean the patient never had the condition.
PULSNAR learns from patients with diagnosis codes and then estimates how many similar cases might be present among those without codes. This approach recognizes that some cases are more likely to be coded than others, and doesn’t assume coded cases are random.
Praveen Kumar, PhD, the study’s first author, said, “Medical records can make self-harm hard to see in more than one way. Sometimes the history is in a clinician’s note but not in the diagnosis codes. Other times, the record may contain risk factors, injuries, poisonings, or behaviors that are consistent with self-harm, even though the record alone does not prove what happened or why. Our method can help flag both patterns for review. This study could verify the first pattern, because the evidence was already in the notes. The second pattern may be just as important, but confirming it would require talking with patients or using information beyond the medical record.”
The team brought together experts from the UNM Health Sciences Center, Raymond G. Murphy Veterans Affairs Medical Center, Vanderbilt University Medical Center, and other institutions, combining knowledge from medical informatics, computer science, psychiatry, economics, and health services research.
A closer look at what’s next for mental health care
This study is part of a broader effort to use machine learning to find health conditions that are often under-recorded in medical data. The team has already used similar methods to uncover under-coded opioid use disorder and is working on expanding this approach to conditions like PTSD, depression, bipolar disorder, and sleep disorders.
While the method is still in development and not yet ready for clinical use, it shows promise for helping health systems get a clearer picture of mental health needs. By identifying documented history that is hard to spot, this technology could help care teams find patients who may need more support.
As Lambert stated, “Self-harm history matters too much to stay buried in records that are not practical to review line by line during routine care. Our work is about helping researchers and health systems find documented history and clinically relevant patterns in the data, so care teams can have a more complete picture of the people they serve.”
Better measurement tools like this could help health systems plan more effectively, improve research, and ultimately lead to better care for those at risk of self-harm and related conditions.
Source: News Medical