Contributed Commentary by Jack
Modell
December 5,
2014 | During
my 20 years in clinical practice as a psychiatrist and medical researcher, I
noticed early on among my patients with major mental illnesses, particularly
those with severe depression and psychotic disorders, that they did not
generally get better by my simply being nice to them, treating them with
ineffective medications, seeing them weekly for office visits, or by providing
other so-called supportive interventions that did not directly address the
underlying illness. This is not to say that kindness and supportive therapy are
not critical to the patient-physician relationship, but rather that kindness
and support alone rarely make a biologically based illness substantially improve
or disappear.
With this background, I vividly recall my surprise when
I joined the pharmaceutical industry and was asked to figure out how to
decrease the nearly 50 percent placebo-response rate they see in antidepressant
trials for major depressive disorder. I had a hard time believing that 50
percent of patients in a true major depressive episode get better on placebos
or just by seeing the doctor every couple of weeks, but they had voluminous
data supporting their figure.
Reviewing the data and published explanations for high
“placebo response rates” in psychiatric studies, three particularly important
factors emerge which are easily mitigated by proper trial design and training.
1) Subjects admitted into clinical trials often have psychiatric
symptoms, but do not meet criteria for the disorder being studied. Examples
include subjects with personality disorders whose symptoms partly mimic the
disorder being studied, subjects with symptoms in response to a particular
stressor (not of sufficient severity or duration to meet formal criteria for a
major psychiatric disorder and likely to abate over time), and subjects who may
feign or exaggerate symptoms to gain attention or access to a clinical trial. Unlike
the very sick patients I encountered in clinical practice, subjects with these
presentations often improve with only supportive interventions.
Recruitment of severely ill subjects is difficult because of widespread
availability of effective medications. Patients in a major depressive episode,
for example, are reluctant to commit to the effort, uncertainties, and
treatment delays involved with a clinical trial when a prescription for an
effective antidepressant can be filled quickly.
2) Investigators often inflate or deflate ratings to
enable subjects to enroll, or remain enrolled in a clinical trial. This can be
done by coaching subjects on their answers or, when subject responses fall in
between scale severity ratings, by rounding to a rating that is more likely to
qualify the subject for the trial.
At the first follow-up visit, with less incentive to influence rating scores,
the scores of the entire included population return toward their true values. Moreover,
subjects and investigators, expecting that the onset of treatment should
coincide with some clinical improvement, may bias rating scores to reflect this
expectation even though the symptoms of the illness may not show much true
change.
While this early “improvement” in rating scores for subjects in clinical trials
may appear to be a placebo effect, it is largely driven by artificially
inflated scale scores regressing back to their true distribution. The
introduction of non-qualified subjects to the study and rater bias will
continue to hamper detection of actual drug-placebo differences throughout the
study.
3) Investigators do not always understand the objective
of the clinical trial, which should not be “to show treatment efficacy,” but
rather to test the null hypothesis of no treatment difference or to estimate
likely treatment effect, as well as objectively recording all adverse events. Many
also do not fully understand the importance of objectivity and consistency in
performing clinical ratings, the intention and importance of inclusion and
exclusion criteria, and the destructive effect on the outcome and scientific
integrity of the trial that including subjects who are not fully qualified can
have.
Each of these factors can skew drug and placebo trial
populations and results, making it appear that subjects improved well beyond
what would have resulted with strict adherence to protocol requirements and
objective assessment of study entry and outcome measures.
What can be done to prevent these problems? Thorough
and meticulous investigator and rater education and training are essential. Often,
perfunctory explanations of the protocol and clinical assessment tools are
provided at investigator meetings, and rater training takes the form of brief
demonstrations of how the rating scales are used, without testing raters to ensure
they understand how the scales are to be used, scored, and interpreted. Rater training
must include testing for adequate understanding and proficiency in use of each
of the study measures.
Beyond review of the protocol design and study
requirements, training of research staff must include detailed explanations
about why the trial is designed exactly as it is; the importance of adherence
to inclusion and exclusion criteria; and the need for honesty, objectivity, and
consistency. Detailed training on the disease under study also ensures that
site staff have a complete understanding of the intended clinical population
and its characteristics.
No matter how well-informed or well-intentioned
investigators might be, humans cannot match computers in objectivity and
consistency. Unless programmed to do so, a computer cannot coach a subject on
how to respond, nor inflate (or deflate) ratings based on feelings,
expectations, or desired outcomes. Following precise algorithms, a computer faithfully
asks and interprets questions and records their responses the same way every
time. Several studies have shown that computerized assessments of entry
criteria and outcome measures in clinical trials – in particular, interactive
voice/web response systems (IVRS/IWRS) – provide data of quality and
signal-detection ability that meet or exceed that obtained by human raters. Strong
consideration should be given to using IVRS/IWRS systems for assessing study
entry criteria and endpoints that allow such use.
The author gratefully acknowledges John H. Greist, MD,
for his outstanding research and input.
Jack Modell, M.D., is the senior medical officer at
Rho. A board-certified psychiatrist with more than 30 years of experience in
clinical research, teaching and patient care, he has also spent nearly 15 years
working in clinical drug development. Modell’s specialties include
neuroscience, pharmacology, drug development, clinical research, medical
governance and clinical diagnosis and treatment. He is known for leading the
first successful development of preventative pharmacotherapy for the depressive
episodes of season affective disorder.
Selected References
Greist JH, Mundt JC, Kobak K. Factors contributing to failed trials of new
agents: can technology prevent some problems. J Clin Psychiatry 2002;63[suppl 2]:8-13.
Mundt JC, Greist JH, Jefferson JW, Katzelnick DJ,
DeBrota DJ, Chappell PB, Modell JG. Is it easier to find what you are looking for if you
think you know what it looks like? J
Clinical Psychopharmacol 2007;27:121-125.
http://www.healthtechsys.com/publications/ivrpubs2.html