Forecasting Trial Enrollment - More Data, Better

Whitepaper
Forecasting Trial Enrollment: More Data,
Better Analytics, Greater Predictability
Feasibility
WHITEPAPER
Forecasting Trial Enrollment: More Data, Better Analytics,
Greater Predictability
1
It is the nature of clinical research in the life sciences to deal in big questions
and great uncertainty. Is the product safe? Will it perform effectively? At
what dose? In what population? With what risks?
Yet one aspect of clinical trials presents sponsor companies with unnecessary
uncertainty: patient recruitment. Sponsors—and the clinical research
organizations (CROs) working on their behalf—are often caught off guard by
the difficulties they experience in enrolling subjects into trials. This is more
than an unpleasant surprise; it is one of the leading causes of trial delays and
even of trial failures.
Using a data-driven approach to determining trial feasibility can all but
eliminate the guesswork surrounding patient recruitment, specifying with
a high degree of confidence exactly how long it will take to fulfill a study’s
patient quota. The study forecast can be created on the basis of where the
right patients can be found, which investigator sites should be involved, and
what the impact of patient outreach will likely be.
With such foreknowledge, companies can make prudent investment decisions,
enter trials with realistic expectations, and improve their trial success rates.
Build It and They Will Come … or Will They?
Over time, life sciences companies have adopted a build-it-and-they-will-come
approach to clinical trials. The premise, not unwarranted given the industry’s
historical success, has been that with the right protocol and the right investigator
sites, patient enrollment is accomplished easily enough. Unfortunately, that thinking
is a carryover from simpler times and no longer holds true.
1 “Web-Based Patient Recruitment: Best Opportunity to Accelerate Clinical Trials,” Cutting Edge Information, Durham, NC.
2 Impact Report, Tufts Center for the Study of Drug Development, Vol. 15, No. 1, Tufts University, 2013.
I NVENTIV C LI N ICAL TR IAL R EC RU ITMENT SOLUTIONS
|
FEASI B I LITY
|
January 27, 2014
2
Feasibility
• Nearly 80 percent of clinical trials fail to meet enrollment timelines
•
1
Approximately a third (30 percent) of Phase III study terminations are due
2
to enrollment difficulties , making recruitment the single biggest reason
for trial failure
Why is this so? There are,
of course, a host of factors
that make patient enrollment
genuinely difficult and arguably
the most challenging step in
running a clinical trial. (See
“Patient Enrollment: It’s a Tough
Job.”) However, the root of
the problem is that typically,
companies embark on trials with
a false sense of what to expect;
their trial forecasts are faulty, and
so it is therefore no wonder that
reality diverges from the plan. In
other words, the issue is not that
a trial might, for example, take
28 months to enroll the required
number of patients, but rather
that it was forecasted to take
only 20.
WHITEPAPER
Two facts surrounding trial recruitment today prove that point all too well:
Patient Enrollment—It’s a Tough Job
Obviously, the number of eligible patients—and
their motivation and willingness to participate in
a trial—affect the ease and speed with which
subjects can be found and enrolled. Below
is an abbreviated list of the factors influencing
patient availability, only some of which are
within a sponsor’s power to control:
•
The epidemiology of the patient
population. How many people suffer from
the condition?
•
The level of unmet need in the market.
How successful are existing therapies
in controlling/curing the condition in
question?
•
The protocol design. How restrictive are
the inclusion/exclusion criteria?
•
The competitive landscape. How many
trials will be running simultaneously using a
similar patient population?
•
Patient knowledge of the trial. Patients
who are not made aware of a trial—either
by their physicians or via broadcast and
social media—are clearly not in the running
to participate in it.
“Sponsors and CROs pay a
heavy price for deviating from
their enrollment forecasts,”
says Otis Johnson, executive
director of Clinical Informatics
& Feasibility at inVentiv Health,
Inc. “By introducing more predictability into their trial planning process, they can
improve their enrollment success rates and adherence to study timelines—both of
which have a significant upside for managing research costs.”
1 Hess, Jon, “Web-Based Patient Recruitment,” Cutting Edge information, http://www.cuttingedgeinfo.com/process/?ref=122
2 Li, Gen, PhD, MBA and Gray, Robert, MBA, “Performance-Based Site Selection Reduces Costs and Shortens Enrollment Time,” Monitor, December 2011. Based on analysis of 5,000 terminated clinical trials.
I NVENTIV C LI N ICAL TR IAL R EC RU ITMENT SOLUTIONS
|
FEASI B I LITY
|
January 27, 2014
3
Feasibility
Patient motivation is yet another confounding factor. Once recruited, patients may be reluctant to
participate for a wide variety of reasons ranging from simple logistics … to fear … to the inconvenience of
steps in the protocol … or to the invasiveness of treatment and diagnostic procedures. On the flip side,
they may be motivated by an equally wide range of internal and external drivers. According to a summary
of literature and research reports within the National Cancer Institute (NCI), the following are the reasons
patients most often cite for being amenable to trial participation*:
•
Doctor’s influence/recommendation
•
Hope for a therapeutic benefit
•
Altruism or to advance science
•
Lack of other medical options
•
Ability to gain access to leading specialists
•
Ability to receive cutting-edge care and the latest treatment discoveries
WHITEPAPER
Patient Enrollment—It’s a Tough Job (continued)
* Williams, Sandra, “Clinical Trial Recruitment and Enrollment: Attitudes, Barriers, and Motivating Factors.” August 2004.
Enrollment Forecasting on the Back of an Envelope
Because most sponsors primarily rely on investigator sites to carry out trial
recruitment, they, quite logically, assess recruitment feasibility by surveying sites on
their projected enrollment capability. They ask for input via questionnaires on the
number of patients investigators treat who fit the study criteria and how many they
believe they would be able to recruit for an upcoming trial.
On a basic level—and particularly in combination with other information—this
step is certainly worthwhile. One caveat to bear in mind, however, is that most
physicians do not actually run queries against a patient database to answer the first
question with any precision; they simply provide a rough estimate of their current
patient population. Nor can they divine the future; again they make an educated
guess as to how many patients they would hope to be able to furnish. Thus,
patient counts gathered directly from investigators should be taken for what they
are: a best-guess estimate provided by a busy professional eager to do the best
thing for patients.
I NVENTIV C LI N ICAL TR IAL R EC RU ITMENT SOLUTIONS
|
FEASI B I LITY
|
January 27, 2014
4
Feasibility
WHITEPAPER
The bottom line is that sites tend to over commit to trial proposals and are overly
optimistic about their ability to supply patients. In reality, in any given trial, 11 percent
3
of sites fail to enroll a single patient and 37 percent under enroll .
Another limitation of this step is that frequently, feasibility surveys go unanswered
at investigator sites. Consequently, it is impossible to know how many potentially
productive sites are overlooked.
Traditionally, companies have tempered the results of such investigator surveys
with their own intuition and judgment to estimate how long it will likely take to
recruit the necessary number of target patients for a study. How well this back-ofthe-envelope forecasting method works depends, of course, on the experience of
those involved. At times, it may produce a valid estimate, but at others it may be
wildly inaccurate, leading to variations in recruitment success.
Multi-Dimensional Enrollment Forecasting
There is nothing wrong with the above steps as far as they go; they simply stop
well short of being sufficient to consistently produce accurate enrollment forecasts.
Fortunately, today there are many more rich sources of data that can inform the trial
forecast, as well as sophisticated statistical tools to test various “what if” scenarios
and establish confidence levels for the results.
The current best-practice approach is to incorporate information gleaned from
physician surveys and the experienced judgment of study planners as described
above into a much more comprehensive, data-driven analysis of enrollment
potential. The result is a baseline forecast that predicts the probability of enrollment
success in a specific timeframe, given certain variables, with a high degree of
accuracy.
While taking a data-driven approach necessitates consulting a variety of data
sources, the process does not take long if the proper sources are already identified
and the analytical steps and statistical methodologies already established. It
is possible to prepare a baseline enrollment forecast in a few days, particularly
as some of the research steps can be performed concurrently. “Life sciences
companies have to live with any number of unknowns as they embark on clinical
trials, but they can have a realistic idea of how best to enroll patients and how long
it will take,” assures Johnson.
3 Impact Report, Tufts Center for the Study of Drug Development, Vol. 15, No. 1, Jan/Feb 2013.
I NVENTIV C LI N ICAL TR IAL R EC RU ITMENT SOLUTIONS
|
FEASI B I LITY
|
January 27, 2014
5
Feasibility
WHITEPAPER
“A little due diligence and patience at the planning stage is just the ‘ounce of
prevention’ that companies need to, in the long run, save time, resources, and
headaches—headaches that can run into millions of dollars very quickly.”
Two main tracks of research should be undertaken to 1) Assess the number of
patients that meet the study criteria and 2) Select those sites that will deliver the
most patients.
Assessing Patient Availability
The first step of the process is to estimate the number of patients who are eligible
to participate, based on the epidemiology of the population by geography, the
study inclusion/exclusion criteria, the treatment guidelines and procedures, and the
competitive landscape. The competitive landscape can by surveyed by consulting
clinical trial registries, publications, and subscription databases. It is possible to
analyze the inclusion/exclusion criteria against data in Electronic Medical Records
(EMRs), pharmacy, or integrated medical claims databases to determine the
proportion of patients that meet the criteria and to pinpoint where they’re located.
These databases can be searched by diagnostic and procedure codes as well as
medication restrictions, yielding a very refined count of potentially eligible patients.
The output can be viewed in a Heat Map, such as is shown in Figure 1, illustrating
the prevalence of patients in the target population, by location.
Figure 1: Prevalence Rate of Alzheimer’s Patients
“The issue is not that a trial might, for example, take 28 months to enroll the
required number of patients, but rather that it was forecasted to take only 20.”
I NVENTIV C LI N ICAL TR IAL R EC RU ITMENT SOLUTIONS
|
FEASI B I LITY
|
January 27, 2014
6
Feasibility
The goal of this phase of the research is to be able to zero in on those sites
that have the ability to deliver the most patients based on their access to target
patients, historical performance, capabilities and resources.
Determining sites’ proximity to patients is a matter of overlaying the Food & Drug
Administration’s (FDA’s) 1572 database of clinical investigators onto a patient Heat
Map. This will readily highlight those investigators in close proximity to clusters of
patients who match the inclusion/exclusion criteria.
WHITEPAPER
Selecting the Most Productive Sites
“Patient counts gathered directly from investigators should be taken for what
they are: a best-guess estimate provided by a busy professional eager to do the
best thing for patients.”
Evaluating sites based on their prior performance can involve, at the most basic level,
reviewing historical data on analogous products from within a company’s own clinical
trial management system and reviewing public sources such as clinicaltrials.gov. As
internal sources are limited in scope and public sources are limited in detail, the ideal
solution is to also tap a commercial database maintained expressly for this purpose.
Through such a service, investigators are assigned an objective, composite performance
score based on the number of trials they’ve participated in and their past enrollment
performance, including initiation periods, screening rates, and failure rates.
Yet another component of identifying suitable sites is to understand the disease-specific
capabilities of individual research centers. Again, this information is commercially
available through a powerful analytical tool that profiles sites on a range of dimensions
including their research activity, infrastructure, personnel, and timelines.
2
Simulating Enrollment Rates
The results of the above research can be loaded into a statistical model designed
to forecast enrollment rates based on different variables.
I NVENTIV C LI N ICAL TR IAL R EC RU ITMENT SOLUTIONS
|
FEASI B I LITY
|
January 27, 2014
7
Feasibility
WHITEPAPER
(Note: the actual inputs need to be customized for every analysis; not all are
appropriate for every situation.) The specific statistical technique, Monte Carlo
Simulation, calculates the probability that a particular outcome will occur based on
a given action or set of assumptions. It works by assigning a range of values to
every input variable and then running a series of virtual trial simulations (anywhere
from 1,000 to 5,000), each using a different set of random values from within the
range.
The results can be displayed as a distribution chart showing the probability of a
given outcome—in this case meeting enrollment targets—for each scenario. Figure
2 illustrates how the probability of enrolling a trial is distributed across time. Here
we see that there is a 10 percent probability of enrolling the trial in 9.7 months and
a 90 percent probability of doing so in 18 months.
Figure 2: Enrollment Probability by Month
This level of precision in enrollment forecasting has never been possible before
and stands to dramatically improve the trial enrollment process by setting realistic
expectations from the outset. “This new application of statistical modeling is a
breakthrough for study planners,” contends Johnson. “They’ve been accountable
for something that often spins out of control. With so much data available with
which to make projections, they can now approach their work with confidence,
knowing that they can recruit according to forecast and drastically limit the amount
of ‘bad news’ they have to deliver.”
I NVENTIV C LI N ICAL TR IAL R EC RU ITMENT SOLUTIONS
|
FEASI B I LITY
|
January 27, 2014
8
Feasibility
WHITEPAPER
3
Calculating the Value of a Direct-to-Patient Enrollment Campaign
The analysis on the previous page was conducted without factoring in how
a patient recruitment campaign could boost recruitment rates and/or reduce
recruitment timelines. This variable deserves special mention because many
maintain that patients’ lack of awareness heads the list of hurdles to achieving
4
recruitment targets . Plus, it is one of the aspects of enrollment that lies within
sponsors’ power to control.
Traditionally, most sponsors have built their overall trial budgets without any
consideration for patient recruitment. This may stem from the fact that there’s been
a longstanding separation of the clinical and marketing functions in life sciences
companies—for valid reasons. Thus, clinicians, not being marketers, don’t
usually focus on how communication channels can be used to increase patients’
awareness of clinical trials. (To make up for this, a few progressive companies
have created specially staffed patient recruitment teams within their clinical
departments—a model that has much merit.)
As a result, recruitment campaigns are often bolted onto trial plans late in the
game—for example when a trial is in rescue mode. This does outreach programs
a disservice because they can be very effective in speeding recruitment timelines
when they are conceived early and funded properly. The range of techniques used
successfully include using broadcast media, digital platforms and social media for
direct outreach to patients as well as providing patient support and awareness
materials to physicians and pharmacists.
“It is possible to produce a baseline forecast that predicts the probability of
enrollment success in a specific timeframe, given certain variables, with a high
degree of accuracy.”
Determining how to allocate communication resources to recruit patients is itself a
well-developed science that is beyond the scope of this paper. It is worth noting,
however, that outreach plans are the result of extensive research into media reach,
audience profiles, and performance benchmarks. Ultimately, recommendations on
which media to use, how often, where, and in what way are very data driven.
4 Williams, Sandra, “Clinical Trial Recruitment and Enrollment: Attitudes, Barriers, and Motivating Factors,” A summary of Literature and Market Research Reports Held by NCI, August 2004.
I NVENTIV C LI N ICAL TR IAL R EC RU ITMENT SOLUTIONS
|
FEASI B I LITY
|
January 27, 2014
9
Feasibility
WHITEPAPER
Ideally, the pros and cons of running a recruitment campaign should be considered
from the beginning, when sponsors are creating their trial plans, forecasting
enrollment timelines, and building their budgets. But, how can companies make
objective decisions about the value of such a campaign at the early stages of trial
planning, when so many assumptions about so many variables are being made?
The answer is, once again, to draw upon data and to apply analytical tools to aid
in the decision process. At the same time that the above research into patient
availability and site selection is taking place, a separate historical analysis can
reveal the impact of recruitment campaigns conducted for studies on analogous
products. As before, an individual company would have limited historical data on
which to draw; this analysis requires access to a comprehensive database covering
hundreds of programs across therapeutic areas and geographies.
Based on such data, one can estimate the cost and effort involved in drawing
potential trial subjects to a website or call center to learn more about a trial. One
can further estimate how many of those would pass the initial screening, how many
would be referred to sites, how many would pass the onsite screening, and finally
how many would sign a consent form and actually be enrolled and randomized
(See Figure 3.)
Figure 3: Estimated Patient Counts by Recruitment Stage
I NVENTIV C LI N ICAL TR IAL R EC RU ITMENT SOLUTIONS
|
FEASI B I LITY
|
January 27, 2014
10
Feasibility
WHITEPAPER
While such precise estimates of campaign performance are very valuable, they
should not be viewed in isolation from the work done to estimate enrollment
timelines. In fact, the estimates of campaign productivity can be fed into the above
simulator to provide a revised view of the recruitment timeline. For example, for a
study requiring 1,000 patients, the Monte Carlo Simulation may show that there’s
a 90 percent probability of recruiting those patients in 18 months by working
exclusively through sites. However, because a direct-to-patient campaign can be
expected to deliver another 200 patients, it could, in effect, shave off 20 percent of
the original enrollment timeline.
“Patient awareness campaigns can be evaluated not only on the number of
patients they can produce, but also on their ability to speed enrollment.”
The probability distribution shown in Figure 2 has now shifted, as seen in Figure 4,
and there is now a 90 percent probability of completing enrollment in 14.8 months.
The patient awareness campaign can thus be evaluated not only on the number of
patients it can produce, but also on its ability to speed enrollment.
Figure 4: Enrollment Probability by Month, With Patient Outreach
I NVENTIV C LI N ICAL TR IAL R EC RU ITMENT SOLUTIONS
|
FEASI B I LITY
|
January 27, 2014
11
Feasibility
WHITEPAPER
4
Conclusion
Enrolling patients in clinical trials is no simple matter, and sites’ inability to
consistently live up to sponsors’ expectations is a growing source of frustration
and inefficiency for sponsors and CROs alike. Part of the solution is to begin
with a better plan—one built on realistic enrollment projections as deduced from
comprehensive data sets and sophisticated analytical tools. Another is to bolster
sites’ recruitment efforts with a direct-to-patient campaign, should statistical
modeling suggest that doing so would be a worthwhile investment. Companies
can now, for the first time, weigh the costs of such a campaign, not just against
the number of patients it can be expected to produce, but also in terms of its likely
impact on the enrollment timeline. These analyses can go a long way toward
reducing the unpredictability in trial planning and improving trial success rates.
I NVENTIV C LI N ICAL TR IAL R EC RU ITMENT SOLUTIONS
|
FEASI B I LITY
|
January 27, 2014
12