by A Yale-led study has revealed that the mathematical models currently used to predict treatments in personalized medicine have limited effectiveness. Personalized medicine is an approach in which healthcare professionals tailor treatments to a patient’s unique genetic profile. The study analyzed clinical trials for multiple schizophrenia treatments and found that while the mathematical algorithms were able to predict patient outcomes within the specific trials they were developed for, they failed to work for patients participating in different trials.
The findings, published in the journal Science, challenge the status quo of algorithm development and highlight the need for improved models in the healthcare sector. Adam Chekroud, an adjunct assistant professor of psychiatry at Yale School of Medicine and the corresponding author of the paper, stated, “Right now, I would say we need to see algorithms working in at least two different settings before we can really get excited about it.”
Schizophrenia, a complex brain disorder affecting approximately 1% of the US population, exemplifies the need for personalized treatments. Up to 50% of patients diagnosed with schizophrenia do not respond to the first antipsychotic drug prescribed to them, but it is currently impossible to predict which patients will respond to therapies. Researchers hope that new technologies utilizing machine learning and artificial intelligence (AI) may yield algorithms that can better predict which treatments will be effective for different patients, leading to improved outcomes and reduced costs of care.
However, due to the high cost and time required to conduct clinical trials, most algorithms are only developed and tested using data from a single trial. The researchers aimed to verify whether these algorithms would work when tested on patients with similar profiles receiving similar treatments. By aggregating data from five clinical trials of schizophrenia treatments made available through the Yale Open Data Access (YODA) Project, the researchers discovered that the algorithms effectively predicted patient outcomes within the specific trial they were developed for, but they failed to accurately predict outcomes for schizophrenia patients in different clinical trials.
The study’s lead author, Adam Chekroud, highlighted that most mathematical algorithms used in medical research were not designed to be used with smaller data sets like those in clinical trials. The application of AI tools to analyze smaller data sets often leads to over-fitting, where a model has learned patterns that are specific to the initial trial’s data but disappear when new data are included. Chekroud emphasized the need to develop algorithms in multiple different contexts before they can be trusted, comparing it to the process of developing new drugs.
The researchers also highlighted the potential inclusion of additional environmental variables, such as drug abuse or personal support from family or friends, which can affect treatment outcomes. They noted that most clinical trials use precise criteria and limits on doctors administering treatments, whereas real-world settings have a wider variety of patients and greater variation in the quality and consistency of treatment.
The study’s findings not only address the challenges of personalized medicine in schizophrenia trials but also raise questions about its broader application in fields like cardiovascular disease and cancer. The authors suggest that sharing data among researchers and expanding the collection of data by large-scale healthcare providers could help increase the reliability and accuracy of AI-driven algorithms. The study was co-authored by Hieronimus Loho, Ralitza Gueorguieva, and Harlan M. Krumholz from Yale School of Medicine.
*Note:
1. Source: Coherent Market Insights, Public sources, Desk research
2. We have leveraged AI tools to mine information and compile it
