Contributed Commentary by David Thompson

April 4, 2018 | Drug manufacturers plan Phase II-III research with utmost care, but all too often regard Real World Evidence (RWE) generation in Phase IV as a check-box activity. Analyze some claims data (check), stage a product registry (check), develop an economic model (check), summarize it all in a global value dossier (check). The expectation is that the results of these activities will automatically dovetail with commercialization efforts and support market access, favorable reimbursement, and premium pricing. (Check, check, and check!)

Not surprisingly, this approach is rarely successful. 

Evidence generation leading up to product approval is a relatively uncomplicated proposition. In general, there is one audience: regulatory; there are two types of endpoints: safety and efficacy; and there is one study design in a randomized controlled trial (RCT). In other words, the process of gaining approval can be characterized as involving the design of a series of RCTs that encompass relevant measures of product safety and efficacy that regulatory authorities can evaluate to decide whether or not to grant licensing approval.

Those involved in clinical development will no doubt be up in arms that anyone would characterize this as “relatively uncomplicated.” Fair enough, but consider how evidence needs change as we progress into the post-approval phase and evidence generation evolves from the experimental setting to the real world setting.

First, there is not just one audience but a much broader array of health system stakeholders who demand RWE. Regulatory authorities remain in play, with an interest in continued safety surveillance once the product hits the market but increasingly issuing post-marketing commitments for evidence generation as well. A second major stakeholder is the payer community, including private and public insurers, who utilize RWE to help inform reimbursement, formulary tiering and co-pay decisions. In the current era of patient-centricity, patients and patient advocacy groups represent another stakeholder group interested in RWE. Less so in the U.S., but in many countries health technology agencies, such as NICE in the UK, represent another important constituency. Finally, clinicians, hospitals and others involved in healthcare delivery find RWE useful and thus constitute another stakeholder with needs to be addressed.

Second, there are not just two types of study measures but far more of interest in real world research. Interest in efficacy morphs into an interest in effectiveness, with the latter concept measuring how well drugs work in conditions of typical clinical practice as opposed to idealized experimental conditions in which efficacy is explored. Safety measures continue to be explored in real-world research but so too do patient-reported outcomes (PROs), treatment patterns and adherence with therapy, as well as treatment costs and other kinds of economic measures, such as cost-effectiveness and budgetary impact.

Third, there is not just one study design but a wider variety from which to choose. Retrospective approaches, including manual chart review and database analyses, are widely used and appropriate for certain measures, such as treatment patterns, adherence, and costs of care. Prospective approaches include traditional observational studies or registries as well as the pragmatic clinical trial, which is the real-world analogue to the RCT. Finally, economic modeling is frequently used to address issues of cost-effectiveness and budgetary impact.

So in comparison to clinical research, real world research does indeed appear to be more complicated. But having more stakeholders, more measures and more study designs is only part of the complexity. To make matters worse, healthcare system stakeholders differ in terms of their preferences for RWE and the study designs differ in terms of their suitability to address the different study measures. Complexity begets more complexity in the real world!

There is clearly a delicate balancing act between manufacturers’ supply of RWE and health system stakeholders’ demand. As noted above it’s complicated for a variety of reasons and unlike the market forces of supply and demand that yield equilibrium in the economy, there are no such market forces at work in real-world evidence.

The concept of “evidentiary equilibrium” refers to the process by which manufacturers establish that their evidence generation efforts will meet stakeholder needs. The historical approach to RWE generation involved a linear process in which real-world studies were designed by manufacturers, executed by the research partners, and then disseminated to their stakeholders. This approach is deficient because stakeholder engagement is focused on the back end so that key research design decisions are made with little or no input from those who are expected to utilize the research findings in decision making. Evidentiary equilibrium is unlikely to prevail.

Replacing this linear process with a continuous one is critical to attaining equilibrium—and stakeholder engagement has to occur right up front. Real-world research design is optimized when it is based on solid insights and preliminary buy-in from the health system stakeholders who will be expected to act upon the findings. Involving them in the design process upfront can be accomplished in a variety of ways, including electronic surveys, expert panels, and one-on-one structured interviews.

Following solicitation of stakeholders’ evidentiary needs, studies can be designed to meet these needs directly such that actionable insights can be obtained from the resulting data. Of course, information needs evolve over time so there is no guarantee that evidentiary equilibrium will be achieved, but identifiable trends can be incorporated into the process such that the risk of misalignment between the RWE generated and the RWE needed is minimized.

In summary, manufacturers need to initiate real-world research programs early but NOT as a check-box activity and NOT without first undertaking the advance work of stakeholder engagement to gain alignment between study planning and evidence needs. Investment of time and resources on the front end will reduce the likelihood of any disconnects on the back end. Evidentiary equilibrium is difficult to attain but not at all impossible in the real world research realm.

David Thompson, PhD is Senior Vice President, Real World Evidence Advisory at Syneos Health. David is a health economist with 30 years of experience in the health economics arena, including work in economic modeling, retrospective database analysis, trial-based economic evaluations, and patient-reported outcomes. He is the Editor-in-Chief of Value & Outcomes Spotlight, the journal of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR). David can be reached at david.thompson@syneoshealth.com.