With reports that FDA’s AI Elsa is “confidently hallucinating” studies that don’t exist, the use of AI to streamline drug review and speed up approval is not here yet.
In a July 9
“One of the challenges that came out from the initial release of the Elsa model for FDA is that it was prone to hallucination,” Botha said. “By that, I mean it was making stuff up. … We can't have our AI do that when it comes to critical analysis of core ingredients and component structures that are required. These are elements where a slight deviation makes something safe or not.”
According to a Wednesday report from CNN,1 the problems with Elsa have continued. Quoting six anonymous FDA officials, some still with the agency and others recently departed, the report explained that while the large language model can generate meeting summaries, it has experienced hallucinations by citing studies that don’t exist.
“Anything that you don’t have time to double-check is unreliable,” one of the anonymous FDA officials told CNN. “It hallucinates confidently.”
Elsa is a large language model assistant meant to help FDA employees “read, write, and summarize” faster. The tool can summarize adverse events to support safety assessments, perform rapid label comparisons, and even generate code for internal databases, while running inside a secure GovCloud environment that does not train on industry submissions, protecting that crucial research data.
FDA Commissioner Marty Makary, MD, MPH, unveiled Elsa in a June video,2 saying the agency is using Elsa to “expedite clinical protocol reviews and reduce the overall time to complete scientific reviews.”
Makary said in the video that one reviewer said that a task that took a couple of days now takes 6 minutes. FDA Chief AI Officer Jeremy Walsh said: “Today marks the dawn of the AI era at the FDA with the release of Elsa, AI is no longer a distant promise but a dynamic force enhancing and optimizing the performance and potential of every employee.”
In an interview with CNN as part of their report, Makary acknowledged that Elsa could hallucinate like any large-language model, and that employees are using it for organization and efficiency, and use of it is optional. He added that Elsa will improve, and FDA officials will also get better at using it.
One of President Donald J. Trump’s focus points in 2025 is to reduce the federal workforce, including at the FDA, where
This has led to some slowdowns. In April, the Wall Street Journal reported that the FDA was missing deadlines and ignoring industry messages, resulting in trial delays. In June, the agency missed the PDUFA data for KalVista Pharmaceuticals’ sebetralstat, citing “heavy workload and limited resources,” according to a KalVista news release.3 The drug was since approved on July 15.
On April 1, former FDA Commissioner Scott Gottlieb, MD, weighed in, concerned that the FDA would bring back the days of “drug lag.”
“Through a generation of congressional actions, investments in expertise and hiring, and careful policymaking, we built the FDA into the most efficient, forward-leaning drug regulatory agency in the world—and established the U.S. as the global center of biopharmaceutical innovation,” Gottlieb posted on X,4 formerly Twitter, on April 1. “Today, the cumulative barrage on that drug-discovery enterprise, threatens to swiftly bring back those frustrating delays for American consumers, particularly affecting rare diseases and areas of significant unmet medical need.”
Avoiding these delays is where AI comes into the administration’s plans. HHS Secretary Robert F. Kennedy, Jr. and Makary have beat the drum on using AI such to streamline and speed up the review and approval process. But according to the CNN report, it’s not there yet. Two anonymous FDA staffers said Elsa cannot yet assist with review work.
External observers have also begun to raise concerns. In a client alert, the law firm Hogan Lovells questioned what kind of oversight governs the tool’s outputs, whether benchmarks exist for evaluating its performance, and how “human-in-the-loop” processes are being enforced. “It’s not clear how the agency defines success for Elsa, and whether guardrails are in place to prevent AI-generated errors from influencing regulatory decisions,” the firm wrote.
Makary’s ambitions for AI go far beyond administrative tools. In a June opinion piece in JAMA, Makary and co-author Vinay Prasad, MD, the FDA’s director for the Center for Biologics Evaluation and Research, outlined a vision for “rapid or instant reviews” of drug applications, faster food safety alerts, and the rebuilding of public trust through transparency and innovation. He compared the goals of Elsa and other agency AI tools to the pace of Operation Warp Speed, which helped deliver COVID-19 vaccines in record time.
In his interview with PharmExec, Botha said AI can, theoretically,
“It can look at complex submissions and to do a lot of the grunt work in analyzing ingredients, processes followed, checking for completeness, and other things that a human would historically have to do,” Botha said. “It can also very quickly check for any anomalies that need to be flagged by the human because it doesn't follow protocol or understood protocol based on past experience or data sets.”
But like any large-language model, time and training data is key.
“In order to have a successful AI strategy within the FDA, the organization must start with the training data set for the model being used to build the LLM specific to drug discovery,” Botha explained. “What's the framework that it gives this large language model to reference best past practice, and what are the guardrails being put in place to ensure its not just repeating the status quo from the last 70 years?”
He added: “The benefit of what AI is going to bring to the FDA is still very much on the edge of machine learning.”
