Deborah Borfitz

October 20, 2021 | The reporting of adverse events (AEs) in randomized controlled trials (RCTs) has only modestly improved over the past two decades. But more consistent inclusion of AEs in published studies will not, by itself, provide a balanced picture of the potential harms of interventions, according to Su Golder, Ph.D., associate professor in the department of health sciences at the University of York (England).

Her expertise is systematic reviews of the adverse effects of treatments, which has grown to include the use of unpublished data, text mining, and social media, she says. Data from RCTs are most useful in the aggregate since individual RCTs are “not always very good at looking at harms” due to their size, duration, and inclusion criteria.

How AEs get measured and reported in RCTs is highly variable and, since these are secondary outcomes and journal articles have word limits, mostly appear as supplemental material, Golder says. Even if a study has enough participants to draw statistically significant conclusions, short follow-up periods and the exclusion of people with comorbidities limit the real-world relevance of those findings.

Efficacy of an intervention has long been the focus of RCTs, the “gold standard” for studying causal relationships as randomization eliminates much of the selection bias inherent with other study designs, says Golder. AEs tend to be arbitrarily reported—for example, when their frequency is greater than 5% or those deemed to be “serious” or “significant”—and there are multiple ways to make the calculation and present the results.

In one drug trial reporting few patients experienced erectile dysfunction, the denominator included women, she recalls. Variable results would similarly emerge in oncology trials depending on whether the analysis is across all cancers or on specific cancer types.

Even given the same data and results, researchers can reach entirely different conclusions based on their notion of which AEs are tolerable and which are not, notes Golder. The risk-benefit analysis is itself a subjective exercise. “The only things that aren’t going to have potential harms are things that don’t work.”

AE data is widely available from sources other than RCTs, including patient registries and the Sentinel Initiative of the U.S. Food and Drug Administration, but reported risks are merely correlated with a medicinal product and not proof of causality, Golder continues. But they can be a great way to inform the kind of questions that RCTs ask in the future and are perhaps more meaningful to patients.

While clinicians tend to focus on avoiding serious outcomes such as hospitalizations, disability, and death, patients are generally more worried about symptoms that disrupt their day-to-day life, says Golder. On social media, for example, people taking cholesterol-lowering statins frequently share that they stopped taking the drug because of muscle aches and pain in their legs making it difficult to walk or even get out of bed. 

Adverse events from interventions are thankfully rare, she says, and “there is only so much regulatory [agencies] can do.” Systematic reviews have the benefit of examining all available published and unpublished data on an intervention to pick up any overlooked safety signals, and to enable robust subgroup analysis based on patient characteristics such as age and sex.

Change Agents

Since 2007, Golder has been active with the Cochrane Adverse Effects Methods Group and has co-authored a chapter in the internationally renowned Cochrane Handbook on incorporating adverse effects into systematic reviews using RCTs and, more recently, other types of study designs. The group promotes the inclusion of AE data in those reviews and educates reviewers in how to properly assess adverse effects.

“Often it’s a throwaway sentence like, ‘There were no serious harms,’ and that’s not enough,” she says. Systematic reviews have started to include more unpublished data because of the publication bias favoring studies with positive findings, which is a positive step forward.

Currently, public discourse about COVID-19 vaccines may be putting too much emphasis on AEs and not enough on the potential benefits, adds Golder. Determining the risk-benefit tradeoff is “quite difficult for scientists to do, let alone the public,” particularly when media outlets are emphasizing one over the other.

Golder says she supports movements such as AllTrials, an international initiative calling for all past and present clinical trials to be registered with their full methods and summary reported. With respect to COVID-19 vaccines, full data transparency should be the norm among trial-sponsoring companies and regulatory agencies as well as in both published and unpublished articles, she notes.

In a review of 1,200 RCTs published earlier this year in the Journal of Clinical Epidemiology (DOI: 10.1016/j.jclinepi.2021.04.020), Golder and her colleagues describe current obstacles in the reporting of harms and recommend several updates to the international CONSORT Harms statement. Specifically, they highlight the importance of providing a balanced summary of both efficacy and harm and avoiding multi-component reporting items to facilitate their uptake and usability by study authors. They additionally encourage more scientific journals to mention CONSORT Harms in their submission guidelines.