How to Evaluate PrEP and Vaccines: Urgency for Next-Generation Compounds

Several presentations at R4P 2018 looked at the new challenges of evaluating efficacy of new PrEP and vaccine candidates.

They highlighted the urgency for new trial designs, using new endpoints and statistical approaches, given the ethical need for control arms to include oral PrEP as a standard of care. By definition, this reduces the number of people likely to become HIV positive compared to earlier placebo controlled studies.

Current studies for second generation PrEP either use an active control of oral TDF/FTC (for example, for long-acting cabotegravir injections and for F/TAF), or allow oral PrEP as a choice (as in with the monoclonal antibody VRC01 AMP studies, at least in countries where oral PrEP is approved).

However, when these studies use hard endpoints of HIV infections as the primary endpoint, there non-inferiority becomes difficult to prove if there very few people now become HIV positive.

Instead, this low HIV incidence -- which might easily drop further during the course of a study -- makes traditional study designs unpratical as they would need to enrol much larger numbers of participants or follow them for many more years.

Defining Need and New Approaches

These issues were reviewed by Professor Sheena McCormack from the UK Medical Research Centre (MRC) in one of the plenary talks at the opening of the R4P2018 conference.1

The context for these discussions also include the different levels of evidence required by health commissioners and policy makers, even once clinical efficacy is proven and included in clinical guidelines. While first generation oral PrEP with TDF/FTC is approved by regulatory agencies such as the FDA and the EMA and is clearly effective, safe and cost effective, access remains extremely limited.

While background levels of HIV incidence should fall due to combination prevention including increased HIV testing, linkage to care with universal access to early ART, treatment as prevention, oral PrEP and voluntary medical male circumcision in some settings, HIV eradication on a population level still aims for an effective HIV vaccine as a key long-term goal. Yet the efficacy of current ART arguable sets a higher threshold for vaccine efficacy to likely higher than 50% -- just as it raises the safety threshold for a cure.

One new approach is to use a multi-arm multi-stage (MAMS) study design, and this is already being used for two European vaccine studies: the European HIV Vaccine Alliance (EVHA) and PrEPVacc.

The EHVA study testing three therapeutic interventions -- two vaccines and a monoclonal antibody -- uses an initially low efficacy barrier based on interim lack of benefit to progress to the second stage which a higher barrier to progress.3

The PrEPVacc study is comparing a dual vaccine arm to a triple vaccine arm using a shared control arm that includes short term use of oral PrEP. Again, an interim lack of benefit assessment is used for the first stage is order to progress to the next stage, but this increases to 70% threshold for overall efficacy.4

Both studies are designed to fit the current maximum five-year funding criteria but longer studies using MAMS have the potential to be more efficient at finding effective interventions in shorter times. For example, the still-ongoing STAMPEDE study for prostate cancer originally randomised participants to one of five active arms with a shared standard of care control arm in 2005. In this platform design, less effective intervention arms were steadily dropped and new interventions added. These new study arms were easy to rapidly enrol, and STAMPEDE now reports new efficacy results every 12-18 months that are powered to change standards of care. This design reduces the number of participants using less effective interventions and is flexible to changing standards of care.

Several years ago, an analysis of this approach modelled that some earlier HIV vaccine efficacy studies could have reached results in half the study time.5

Statistical interpretation of increasingly effective PrEP paradoxically can produce datasets with wider confidence intervals for the more effective intervention in similar size studies. One way to overcome these results is to incorporate a projected placebo rate using an inverted infection ratio (AIR). See Table 1.

Table 1: Studies With Fewer Event Generate Wider Confidence Intervals
 Incidence rate/100 yearsIRR





r (n)

New PrEP

r (n)

Study A2 (40)2 (40)1.00




Study B1 (20)1 (20)1.00



(0.88, 1.14)

Based on 2000 PY observation in each arm.

Surrogate Markers for HIV Infection

A second approach to overcome these challenges for PrEP studies was proposed by Jeff Murray from the US FDA who also emphasised that non-inferiority study designs using oral PrEP as an active control are not possible for women because of the lack of a consistently positive treatment effect in earlier PrEP studies.6

Even though non-inferiority study designs can theoretically still be used in gay men, the low expected infection rate would likely either significantly increase study size (>20,000 participants) or significantly extend follow-up time (>7-( years). Either change would make phase 3 studies prohibitively expensive for even the most promising compounds to be studied.

Instead, PrEP studies could perhaps prove efficacy by showing a lack of expected events in a well-defined population in the absence of the intervention. A similar approach has been used to evaluate new oral contraceptives in single arm or active control arm studies using the Pearl index which is calculated based on consistency of pregnancy in population level. Products typically have to show less than two unintended pregnancies per 100 patient years of follow-up, with the EMA requiring a sufficient sample size for the 95%CI for the Pearl Index to not be >1.0.

Even though HIV incidence without an intervention is lower than pregnancy, a similar index for HIV would involve a reliable estimate of expected HIV incidence in populations with significant risk and high prevalence.

For example, rates if indicator infections that correlate closely to likely HIV risk could use rate of rectal STIs. For this to be reliable requires the study population to be at risk for HIV relying on HIV prevalence and behaviour risk.

For example, the close correlation between rectal gonorrhoeae (GC) and HIV incidence in gay men was shown from a systematic literature search that identified nine recent studies in gay men that collected incidence data on HIV and rectal CG. These were used to generate an expected HIV incidence with appropriate 95%CIs for gay men based on background prevalence of GC.

New PrEP interventions for gay men could then use active control arms using current standard of care PrEP and measure efficacy based on rectal GC and other STI endpoints, in population where these rates are already high at baseline. The approach is dependent on defining an acceptable level of background HIV risk (also supported by prevalence at screening) and an assumption that behavioural change is unlikely.

Further Discussions

A third presentation by Deborah Donnell from the Fred Hutchinson Cancer Research Center used examples of current PrEP studies to further expand on new approaches that will be essential now that current approaches to study design are inappropriate or infeasible in most settings.7

Studies using active controls (F/TAF and cabotegravir trials) require all participants to agree to use current standard of care and layer study designs (for VRC01 AMP studies or HVTN vaccine studies) use either placebo or allow use of oral TDF/FTC. In populations with a background HIV incidence of 1%, and similar efficacy, this would require studies involving 40,000 participants with 100,000 person years of follow up.

Examples of potential solutions that were discussed included using averted infections ratio, refining the statistical approach in non-inferiority studies to measure time without incidents, using MAMS or using surrogate endpoints (such as incidence of rectal STIs.


Given the pipeline for PREP is currently very strong and for vaccines is promising, there might be an urgency for regulatory agencies to show rapid flexibility for new ways to validate statistical efficacy.

This might even be needed in time for promising compounds that are currently in phase 3 studies if there are fewer events due to either high PrEP efficacy and/or lower HIV incidence that might have dropped while these studies have been running.

These models will also be included in an upcoming multiagency meeting on PrEP design that will be held in Seattle next month.8


  1. McCormack S. Novel trial designs for a complex environment. R4P2018, 21-25 October 2018. Plenary lecture PL 01.02. (abstract) (webcast)
  2. EHVA study. An HIV Vaccine Trial in Individuals Who Started ART During Primary or Chronic Infection (EHVAT01). NCT02972450.
  3. PrEPVac study. A combination effect study in Africa of two DNA-MVA- or DNA- Env protein HIV-1 vaccine regimens with pre-exposure prophylaxis (PrEP).
  4. Sydes MR et al. Flexible trial design in practice -- stopping arms for lack-of-benefit and adding research arms mid-trial in STAMPEDE: a multi-arm multi-stage randomized controlled trial. Trials 2012, 13:168. DOI: 10.1186/1745-6215-13-168.
  5. Gilbert et al. A sequential phase 2b trial design for evaluating vaccine efficacy and immune correlates for multiple HIV vaccine regimens. Stat Comm Inf Dis (2011) Oct;3(1). pii: 1037.
  6. Murray J. Regulatory perspectives for streamlining HIV prevention trials. R4P2018, 21-25 October 2018. Oral abstract SY03.01. (abstract)
    Webcast link to be added when online (webcast)
  7. Donnell D. Novel trial design in the era of successful HIV prevention interventions. R4P2018, 21-25 October 2018. Oral abstract SY03.02. (abstract)
  8. HIV prevention efficacy trial designs of the future. 5 November 2018.

[Note from TheBodyPRO: This article was originally published by HIV i-Base on Oct. 24, 2018. We have cross-posted it with their permission.]