“Clinical trials are human endeavors, and researchers and regulators need to understand why a model is making a particular recommendation, rather than simply receiving an answer from a black box.”
How AI Is Rescuing Failed Clinical Trials: Q&A With Dr. Joe Geraci
Dr. Joe Geraci of NetraMark explains how AI analysis of clinical trial data can uncover hidden responder subgroups, separate true drug responders from placebo responders, and potentially rescue drug assets that appeared to fail in Phase II.
In a conversation with Pharmaceutical Executive, Dr. Joe Geraci, CSO/CTO and co-Founder of NetraMark, discusses how AI-driven analysis of clinical trial data can uncover hidden patient subgroups capable of turning seemingly failed drug assets into viable candidates for further development.
Geraci explains that clinical trial results can contain meaningful efficacy signals that are hidden within the overall patient population, with subpopulations where a drug is working even when the broader trial does not produce a result strong enough to support a business decision. Using NetraAI, his team identifies these structures within Phase II data to help sponsors determine whether an asset has a population worth pursuing in later trials.
He also addresses placebo response in CNS and psychiatric trials, noting that some people who respond to placebo can also be drug responders, so the relationship is not simply one or the other, which allows researchers to ask whether they can identify patients less likely to respond to placebo while still being good candidates for the drug itself.
On regulatory concerns surrounding AI in clinical trials, Geraci emphasizes that the fundamental issue is transparency, and clinical trials are human endeavors, where researchers and regulators need to understand why a model is making a particular recommendation, rather than simply receiving an answer from a black box. He further describes applying NetraAI to Answer ALS data as functioning like a data microscope: when you zoom out, you see the major clinical groups, but when you zoom in, you can uncover additional structure within them.
A transcript of Geraci’s conversation with Pharmaceutical Executive can be found below.
Pharmaceutical Executive: You've previously spoken about the 90% failure rate in clinical trials. Could you give an example of how going back into Phase II data and isolating true responder subgroups can turn what looks like a failed asset into something worth taking forward into Phase III and beyond?
Dr. Joe Geraci: Clinical trial results can contain meaningful efficacy signals that are hidden within the overall patient population. In many cases, there are subpopulations where a drug is working even when the broader trial does not produce a result strong enough to support a business decision. Using NetraAI, we can identify structures within Phase II clinical data that point to where that advantage exists and characterize those patients in a simple, feasible way.
The goal is to understand what made part of a trial successful, even if the overall trial was considered a failure. Those insights can then be carried into the design of a subsequent trial and help sponsors determine whether an asset has a population worth pursuing.
PE: Placebo response in CNS and psychiatric trials is usually treated as something to control. Why does reframing placebo response matter, and what can separating drug responders from placebo responders reveal that standard trial analysis might miss?
Dr. Geraci: Ultimately, the goal is to identify the broadest population for which a drug works, but placebo response can complicate that analysis when it is modeled incorrectly. From studying placebo response across genetics and personality traits, I've found that some people who respond to placebo can also be drug responders, so the relationship is not simply one or the other.
NetraAI can help characterize patients based on the features associated with their likelihood of responding to placebo and to the drug itself. That allows us to ask a more useful question of whether we can identify people who are less likely to respond to placebo while still being good candidates to respond to the drug. This is essentially a new take on counter-factual modelling. The resulting information can help reveal advantages within the patient population that may inform the design of a subsequent trial.
PE: What do you think are the core objections the FDA and European Medicines Agency might have to black-box AI models, particularly when they are used to identify or select clinical trial patients?
Dr. Geraci: The fundamental issue is transparency. Clinical trials are human endeavors, and researchers and regulators need to understand why a model is making a particular recommendation, rather than simply receiving an answer from a black box. If an AI model says that a particular set of characteristics identifies the right patients for a trial, you need to know what those characteristics are and how the model arrived at them.
Clinical trials are already extraordinarily complex because they combine biological complexity with differences across trial sites and patient interactions. To control that complexity and maintain a scientifically valid experiment, we need models that can explain the parameters they are using, while being powerful enough to discover the non-linear relationships between these factors.
PE: NetraAI was applied to public multi-omics transcriptomic datasets from the Answer ALS initiative. Did its findings change how researchers should think about ALS, and how might it affect the design of trials for other conditions where a single diagnostic label may be masking several different diseases?
Dr. Geraci: ALS illustrates why we need to look at disease biology at multiple scales. Even within sporadic ALS, where the underlying mechanisms can differ, we found clear signals separating patients based on how the disease progresses, including whether it begins in the limbs or in the bulbar region. What was particularly important was that our approach revealed additional structure within those broad categories that other methods did not identify. I think of this as a data microscope: when you zoom out, you see the major clinical groups, but when you zoom in, you can uncover additional structure within them. That kind of analysis can help researchers better understand heterogeneity and design trials around the biological and clinical differences that exist within a single disease label.
PE: If the tools to find hidden responder subgroups and decompose placebo response already exist, why are so many sponsors still running trials the same way and potentially writing off assets that might be recoverable?
Dr. Geraci: The issue is not that sponsors are ignoring stratification or failing to use sophisticated methods. Pharmaceutical companies have highly skilled statisticians and machine learning experts who apply the methods that have been established through research and practice. The challenge is the inertia around the belief that these are the tools that should be applied to these problems.
At the same time, advances in mathematics and computer science are producing new approaches that are not yet widely adopted in clinical development. We've demonstrated in our publications and upcoming work that our methods can outperform established approaches in certain applications. The technology and the methods are maturing, and that creates an opportunity to extract more information from clinical trial data than conventional approaches may reveal. The complexities inherent in patient population data provides very serious challenges to the methods that are commonly used, and this has not become clear to everyone in the field.
PE: How do you handle the smaller datasets that are typical of clinical trials, and have you been able to demonstrate models that actually replicated in real life?
Dr. Geraci: Small datasets are a real challenge for AI, but they can still be useful when the underlying effect size is large enough. Our approach with NetraAI is designed to distinguish between patients the model can understand and those it cannot, allowing it to identify high-effect-size subpopulations rather than forcing every patient into a classification.
In a small dataset, some patients may not resemble anyone else, while larger datasets make it easier to find representative matches. We have also seen replication in practice. In one depression study involving a failed trial, we identified responder subpopulations from the available data and those findings subsequently replicated in two Phase III trials. We have additional work on this that we expect to publish, but the depression study provides an example of how useful signals can be recovered even from relatively small clinical datasets. This is a new paradigm of algorithm that we have access to and it is essentially powerful enough to admit when it doesn’t know, while seeing the way that patients relate to each other without the bias of previous findings. Every trial is unique.
Related to this article








