Will covid-19 vaccines save lives? Current trials aren’t designed to tell us

The world has bet the farm on vaccines as the solution to the pandemic, but the trials are not focused on answering the questions many might assume they are. Peter Doshi reports

Linked Editorial

BMJ 2020; 371 doi: https://doi.org/10.1136/bmj.m4037 (Published 21 October 2020) Cite this as: BMJ 2020;371:m4037

Peter Doshi, associate editor

pdoshi@bmj.com

Covid-19 vaccine trial protocols released

As phase III trials of covid-19 vaccines reach their target enrolments, officials have been trying to project calm. The US coronavirus czar Anthony Fauci and the Food and Drug Administration leadership have offered public assurances that established procedures will be followed.1234 Only a “safe and effective” vaccine will be approved, they say, and nine vaccine manufacturers issued a rare joint statement pledging not to prematurely seek regulatory review.5

But what will it mean exactly when a vaccine is declared “effective”? To the public this seems fairly obvious. “The primary goal of a covid-19 vaccine is to keep people from getting very sick and dying,” a National Public Radio broadcast said bluntly.6

Peter Hotez, dean of the National School of Tropical Medicine at Baylor College of Medicine in Houston, said, “Ideally, you want an antiviral vaccine to do two things . . . first, reduce the likelihood you will get severely ill and go to the hospital, and two, prevent infection and therefore interrupt disease transmission.”7

Yet the current phase III trials are not actually set up to prove either (table 1). None of the trials currently under way are designed to detect a reduction in any serious outcome such as hospital admissions, use of intensive care, or deaths. Nor are the vaccines being studied to determine whether they can interrupt transmission of the virus.

Evaluating mild, not severe, disease

In a September interview Medscape editor in chief Eric Topol pondered what counts as a recorded “event” in the vaccine trials. “We’re not talking about just a PCR [polymerase chain reaction test]-positive mild infection. It has to be moderate to severe illness to qualify as an event, correct?” he asked.8

“That’s right,” concurred his guest, Paul Offit, a vaccinologist who sits on the FDA advisory committee that may ultimately recommend the vaccines for licence or emergency use authorisation.

But that’s not right. In all the ongoing phase III trials for which details have been released, laboratory confirmed infections even with only mild symptoms qualify as meeting the primary endpoint definition.9101112 

In Pfizer and Moderna’s trials, for example, people with only a cough and positive laboratory test would bring those trials one event closer to their completion. (If AstraZeneca’s ongoing UK trial is designed similarly to its “paused” US trial for which the company has released details, a cough and fever with positive PCR test would suffice.)

Part of the reason may be numbers. Severe illness requiring hospital admission, which happens in only a small fraction of symptomatic covid-19 cases, would be unlikely to occur in significant numbers in trials. Data published by the US Centers for Disease Control and Prevention in late April reported a symptomatic case hospitalisation ratio of 3.4% overall, varying from 1.7% in 0-49 year olds and 4.5% in 50-64 year olds to 7.4% in those 65 and over.13 Because most people with symptomatic covid-19 experience only mild symptoms,14 even trials involving 30 000 or more patients would turn up relatively few cases of severe disease.

In the trials, final efficacy analyses are planned after just 150 to 160 “events,”—that is, a positive indication of symptomatic covid-19, regardless of severity of the illness.

Yet until vaccine manufacturers began to release their study protocols in mid-September, trial registries and other publicly released information did little to dispel the notion that it was severe covid-19 that the trials were assessing. Moderna, for example, called hospital admissions a “key secondary endpoint” in statements to the media.15 And a press release from the US National Institutes of Health reinforced this impression, stating that Moderna’s trial “aims to study whether the vaccine can prevent severe covid-19” and “seeks to answer if the vaccine can prevent death caused by covid-19.”16

But Tal Zaks, chief medical officer at Moderna, told The BMJ that the company’s trial lacks adequate statistical power to assess those outcomes. “The trial is precluded from judging [hospital admissions], based on what is a reasonable size and duration to serve the public good here,” he said.

Hospital admissions and deaths from covid-19 are simply too uncommon in the population being studied for an effective vaccine to demonstrate statistically significant differences in a trial of 30 000 people. The same is true of its ability to save lives or prevent transmission: the trials are not designed to find out.

Zaks said, “Would I like to know that this prevents mortality? Sure, because I believe it does. I just don’t think it’s feasible within the timeframe [of the trial]—too many would die waiting for the results before we ever knew that.”

Stopping transmission

What about Hotez’s second criterion, interrupting virus transmission, which some experts have argued17 should be the most important test in phase III studies?

“Our trial will not demonstrate prevention of transmission,” Zaks said, “because in order to do that you have to swab people twice a week for very long periods, and that becomes operationally untenable.”

He repeatedly emphasised these “operational realities” of running a vaccine trial. “Every trial design, especially phase III, is always a balancing act between different needs,” he said. 

“If you wanted to have an answer on an endpoint that happens at a frequency of one 10th or one fifth the frequency of the primary endpoint, you would need a trial that is either 5 or 10 times larger or you’d need a trial that is 5 or 10 times longer to collect those events. 

“Neither of these, I think, are acceptable in the current public need for knowing expeditiously that a vaccine works.”

FULL STORY HERE: 

https://www.bmj.com/content/371/bmj.m4037