By Deborah Borfitz 

August 15, 2023 | Clinical research stakeholders, most especially study sponsors, are now firmly onboard with the idea of using artificial intelligence (AI) to improve the conduct of clinical trials. The U.S. Food and Drug Administration (FDA) expects increasing use of different AI approaches, including natural language processing (NLP), and is actively “inviting different use cases around it,” according to Jeff Elton, Ph.D., CEO of technology development and evidence generation company ConcertAI. 

All the work being done by ConcertAI can be described as “decision augmentation,” he says, including helping sponsors match patients to clinical trials and supporting investigators responsible for populating electronic case report forms (eCRFs) with data about study participants. “We have provider research sites and study sponsors who are moving their entire clinical research infrastructure into a whole new set of [AI] technology-enabled tools.” 

The potential efficiency and productivity gains for biopharmaceutical sponsors are notable. “We can bring a site up in a fraction of the time it would have taken before... lowering some of the burden on research sites so there’s more time to allocate to the patients themselves,” Elton says. More than half of the data that used to be manually entered into eCRFs can now be fully automated, a figure he expects will climb to 75% in the first quarter of 2024. For sponsors who opt for a full electronic package, source data verification and other current manual processes conducted by sponsors or contracted vendors will no longer be required—potentially as soon as the coming 24 to 36 months, he adds.  

Across the broader AI field, he continues, the pace of change has been brisk—"several multiples faster than typical regulatory cycles”—creating a conundrum for the FDA but also an incentive for “coopetition” between companies. “If you’re not using the most advanced tool then you are going to fall behind.” 

ConcertAI is currently partnered with a biopharma sponsor and large academic center to build an AI model that will complement a therapeutic in a randomized controlled trial and undergo its own prospective validation, says Elton. The algorithm is designed to find patients most likely to respond to the drug and could ultimately become part of the standard of care. 

With another sponsor, ConcertAI is collaboratively developing an AI algorithm that will mine digital pathology slides for features predictive of a rare genetic mutation that is expected to speed the accrual of eligible patients into a clinical trial by many years, he adds. Individuals flagged by the model will get confirmatory next-generation sequencing tests. “This would never have been done even 24 months ago.” 

Many Models 

Oncology, hematology, and urological cancers are the focus of ConcertAI, and it has the largest collection of research-grade clinical data in each of those areas thanks to collaborations with about 100 U.S.-based healthcare providers, says Elton. The company also works closely with the American Society of Clinical Oncology (ASCO) and has a collaborative agreement with the FDA to evaluate the efficacy and safety of current treatments in different groups of cancer patients. 

Given all the data at its disposal, ConcertAI develops a lot of its own software as well as some of its own AI models. The company uses more than 150 AI models, he reports, noting that for radiological interpretation alone about 600 AI-related solutions have already been built. 

“We probably have the most representative data with the least bias of anyone doing work in oncology,” said Elton. But the company also routinely pits its approaches and methodologies against those of third parties and collaborates with competitors who rise to the top.   

ConcertAI’s evidence generation work involves the use of retrospective electronic medical records (EMRs), as well as claims and laboratory data, which it unifies to explore a variety of research questions, Elton says. This includes the identification of specific subpopulations to enroll in first-in-human prospective clinical trials.  

It uses the same data-driven approach to examine the effectiveness of medicines in real-world patients, or to broaden their application to different tumors or patient groups than composed the original trial population, he adds. ConcertAI is also called on to help optimize the use of an approved therapeutic, so it becomes more effective for more patients, an exercise that frequently involves the merging of genomic, clinical, and hematologic data. 

ConcertAI’s approach to retrospective evidence generation is to bring AI-powered technology solutions into medium- to large-sized clinical research settings to augment research staff. The gap between supply and demand for competent staff has been widening for years and the pandemic only exacerbated the situation, Elton says. 

Tools for automating some of the arduous tasks in running studies include software for extracting data from EMRs, including NLP models focused on unstructured data, says Elton. That information then auto-populates eCRFs for the research team’s review, acceptance, and modification before backend publication in an electronic data capture system. 

On the provider side, ConcertAI also works closely with the research community to accelerate innovation via software-as-a-medical device where retrospective images and EMR data are being integrated, he adds. This could result in the development and validation of AI models to support human diagnostic and test interpretation capabilities. 

‘Mutual Discovery’ 

Regional health systems have been using AI for research purposes for at least the past three to five years, Elton notes, initially looking largely at structured data that are immediately machine-readable. In oncology, the data of interest for establishing inclusion and exclusion criteria—radiological interpretation and next-generation sequencing reports and other written narratives—are unstructured and readable by computers.  

All that changed with the development of NLP models that can read clinical terminology, including dozens custom-built for clients by ConcertAI. At the 2023 ASCO annal meeting, the company presented how NLP models can be used to enrich structured real-world data accurately and at scale. 

The idea is never to replace human experts, he stresses, but to ensure they don’t miss things. “We’re trying to take advantage of the fact that [AI] can go through a lot more data a lot more quickly than a typical single individual who has to click through multiple windows, screens, and documents inside the electronic environment seeking answers to questions.” 

Much of ConcertAI’s collaborative work with the FDA has been with the agency’s data science team and discussions extend to the use of real-world data in support of regulatory submissions, including external controls as a comparator arm to clinical trials, says Elton. This is largely a process of “mutual discovery” about what is technically and consistently feasible to deploy and comparing what AI can deliver relative to traditional manual approaches. 

The agency isn’t being prescriptive about the AI approaches it condones and may be hard-pressed to keep pace with rapid-fire change in the field should it issue formal guidance any time soon, he adds. Sponsor companies in any case appear poised to take the lead.  

Meaningful Targets 

In terms of real-world healthcare, Elton says there are multiple use cases where AI is being deployed in the clinical setting—most popularly, to identify the best therapy for a particular cancer patient. But they tend to be narrow in application because validity and accuracy can’t be established when the comparator arm is a single clinician reading clinical guidelines to select a treatment.  

ConcertAI has one of the more broadly deployed clinical AI solutions for radiological image interpretation, he notes. Different versions of the model are used in neurological, cardiovascular, and oncological settings to help doctors pinpoint occluded vessels for more targeted intervention. 

Separately, the company has been working with biopharma sponsors such as Bristol Myers Squibb to accelerate patient identification, consent, institutional review board approval, and contract negotiations for clinical trials, and Janssen to broaden access to trials in new sites and strengthen trial diversity. NLP models are also being used behind the firewall at research sites, complemented by study-specific AI models used to identify potential participants. “We’re trying to get as close as possible to zero false-negatives,” says Elton, noting that multiple models are now starting to meet the bar for noninferiority with human experts. 

Large language models are also in the wheelhouse of ConcertAI and they’re being developed to involve human supervision, so they stay within defined parameters versus many of the newer off-the-shelf models (e.g., GPT-3.5, and GPT-4) that self-train on publicly available data on the internet, he says. NLP models were historically trained during the development process by a cross-functional team of domain experts and, if best practices were followed, did not suffer from any nonsensical or outright false “hallucinations.” 

With models like GPT-3 or 4, hallucinations are a recognized problem. Even with training, Elton says, generative AI alone is not yet suitable for patient-facing clinical applications. “There are no intended deployments for generative AI for anything that would be affecting patient care or directly selecting a treatment.” 

However, ConcertAI is developing ways of allowing generative models to self-train but only on data experts determine they should be exposed to, he adds. In combination with large language model NLP, they could give context to the data—not just speed and scale—in ways prior-generation AI do not to improve accuracy of the output. 

The utility of this pairing is that medical devices can start taking on different roles, such as an AI solution that can convert radiological scans into enhanced images to help identify potential abnormalities. Generative models that self-train “look a little bit like a black box,” something regulatory bodies frown on, so the data they feed on will need to be given context.  

Overall, biopharma companies have become true standard-bearers for what high-quality, high-speed research programs look like, says Elton, who previously served as chief operating officer at Novartis. In terms of the clarity around meaningful targets and the inadequacy of the current state of care, “the bars that I see emerging today are the highest I have encountered over the course of my professional career, something that should be a great source of optimism and motivation.”