Protocol complexity is continuing to climb, adding unplanned time and cost to clinical trials, according to a new report that found the number of deviations and substantial amendments has increased almost four percentage points in the past decade.

The study by the Tufts Center for the Study of Drug Development (CSDD) analyzed data from standard trial protocols across multiple therapeutic areas from 2018 to 2020 and found that the mean number of deviations and substantial amendments (changes that required enrollment suspension, internal approval and ethical review board/regulatory approval followed by reconsent of participants) has increased for all trial phases compared to the prior 2013-2015 period. In addition, 2018-2020 saw an increase in the percentage of protocols with at least one substantial amendment, CSDD found.

“There are so many measures that we use to characterize how complexity is evolving — number of endpoints, procedures, eligibility criteria, countries, sites, these kinds of descriptive metrics,” Ken Getz, CSDD director, told CenterWatch Weekly. “Then there are performance metrics, some that relate to cycle time and others that might relate to the burden of design or the infeasibility of the design. That’s where deviations and amendments are so important.”

“It’s very concrete: You know that a substantial amendment is incredibly disruptive and very costly, so to see that they’re rising over time and a larger proportion of protocols experience them, that’s sort of a byproduct of the level of complexity that we’re managing and how it’s challenging patient participation burden as well as site administration burden of the protocol,” Getz said.

Specifically, more than two-thirds (69 percent) of phase 3 protocols had at least one substantial amendment compared to 66 percent in 2013-2015. In addition, the phase 3 protocols in the 2018-2020 period had 3.3 substantial amendments on average compared to 2.3 in the previous period.

For phase 2 protocols, 78 percent had at least one substantial amendment compared to 77 percent in the years prior, while the average number of substantial amendments increased to 2.7 in 2018-2020 from 2.2 in 2013-2015.

The percentage of phase 1 protocols with at least one amendment saw the greatest increase, rising to 59 percent from just over half of protocols (52 percent) in the previous period. Like the others, the average number of substantial amendments went up as well, to 2.4 amendments per protocol from 1.8 amendments.

The rate of substantial amendments is more pronounced in cancer and rare disease trials, CSDD saw; the study found that the average number of amendments per phase 2 oncology protocol was nearly double (4.3) that of nononcology protocols (2.3). For phase 3, oncology protocols had 4.2 amendments on average compared to 3.0 in nononcology.

Similarly, phase 2 rare disease protocols saw 4.9 amendments on average, more than twice the average number of amendments counted in nonrare disease protocols (2.3). In phase 3, rare disease trials had 3.8 amendments on average compared to 3.2 in nonrare disease trials.

Protocol amendments come with costly consequences; for example, previous research by CSDD determined that just one phase 3 protocol amendment costs $535,000 in direct unbudgeted costs and tacked on three additional months of unplanned implementation time. And CSDD found in its most recent study that protocols with three or more substantial amendments took seven times longer on study close out than planned compared to protocols that had two or fewer amendments.

Phase 3 trials were found to have the highest number of deviations per protocol on average (118.5); these deviations also affected the greatest proportion of participants (32.8 percent). Phase 2 protocols saw 75.3 deviations on average, and these also impacted a large number of subjects (30 percent). Phase 1 trials saw the lowest average number of deviations (8.7) and degree of impact (15.3 percent of participants affected).

CSDD noted that there was a high degree of variation in the average number of deviations per protocol across all trial phases, highlighting how difficult it is to anticipate and handle changes in study design and procedures while the trial is being conducted.

Additionally, oncology trials saw significantly higher deviations on average, the center found in its analysis. Combined, phase 2 and 3 oncology protocols averaged 30 percent more deviations compared to nononcology protocols. These deviations also involved 40 percent of participants, almost double that in nononcology protocols. Rare diseases, on the other hand, saw a lower average amount of protocol deviations in a smaller proportion of participants compared to nonrare disease protocols.

Notably, CSDD found that phase 2 and 3 protocols outsourced to full-service CROs had a greater average amount of deviations and substantial amendments per protocol. The average number of deviations per protocol was 64 percent higher than the average number of protocols managed by internal teams — and the portion of trial subjects impacted by these deviations was 71 percent greater. Phase 2 and 3 protocols managed by CROs also had 25.9 percent more substantial amendments.

Don’t be too quick to blame CROs, Getz cautioned, saying he believes that the historic and continued outsourcing of complex protocols is responsible for the higher figures.

“When companies outsource, they keep the simpler ones, the easier ones, for their own in-house teams, and give the really hard ones to the CRO. Those protocols that are more complex and demanding, those are going to have more deviations and amendments because the sites have more difficulty,” Getz explained. “Patient recruitment and patient participation is so much more demanding.”

“The other thing that’s really interesting,” he said, “is that CROs, compared to sponsors, often have better insight into operating data, and they’re looking at it even more frequently because that’s a source of their competitive advantage, so they’re quicker to identify a pattern in the administration of the protocol and to try to act on it quickly.”

While the findings don’t come as a serious shock, given that increasing trial complexity has gained high visibility as one of the most serious and impactful problems in clinical research, they’re still concerning, Getz said, and the increased deviations and amendments have caught the attention of industry. Companies are increasingly gathering feedback from sites and patients and using various tools to conduct burden evaluations, for instance, and they’re bracing for even more deviations in the future.

“With the pandemic and the transition to more virtual and remote-administered protocols, the expectation is that you’re going to see even more deviations. There’s a belief that we’re in an environment that’s going to be just deviation-rich, especially hybrid trials,” Getz said.

“The message here is that there are ways we can spend more time up front, before we actually implement the protocol, where we’re soliciting input from patients with regard to the feasibility, [and from] investigative site professionals to give us insight into places where they think there’s going to be significant burden. There’s also the whole area of site and burden assessment using software tools and burden scores and that sort of thing,” he said.

Going forward, Getz believes that the findings from this latest study, which extend right up to the start of the pandemic, will serve as a valuable baseline. In its next analysis, CSDD intends to compare the impact of the pandemic to its baseline data to see how trial complexity has been affected.

Gain access to CSDD’s Impact Reports here: https://bit.ly/3q9A5tf.