Measures of how well a vaccine works
BMJ 2024; 386 doi: https://doi.org/10.1136/bmj.q1982 (Published 12 September 2024) Cite this as: BMJ 2024;386:q1982Linked Research
Effectiveness of modified vaccinia Ankara-Bavarian Nordic vaccine against mpox infection: emulation of a target trial
- Sharmistha Mishra
, scientist123456, - Christine Navarro, public health physician278,
- Jeffrey C Kwong, senior scientist24578910
- 1Department of Medicine, Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
- 2ICES, Toronto, ON, Canada
- 3MAP Centre for Urban Health Solutions, Li Ka Shing Knowledge Institute, Unity Health Toronto, Toronto, ON, Canada
- 4Dalla Lana School of Public Health, University of Toronto, Toronto, Canada
- 5Institute of Medical Science, Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada
- 6Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada
- 7Public Health Ontario, Toronto, ON, Canada
- 8Centre for Vaccine Preventable Diseases, University of Toronto, Toronto, ON, Canada
- 9Department of Family and Community Medicine, University of Toronto, Toronto, ON, Canada
- 10University Health Network, Toronto, ON, Canada
- Correspondence to: S Mishra sharmistha.mishra{at}utoronto.ca (or @mishrash on Twitter)
When evaluating how well a vaccine works, two categories of outcomes are of interest: infection related outcomes, sometimes referred to as infection blocking effects of vaccines, and disease related outcomes, often referred to as symptomatic disease, morbidity, or mortality outcomes.1 These two categories can then be further divided to reflect those in whom the outcomes are measured: the individual who received the vaccine (direct effects), among contacts of the person who received the vaccine (indirect effects), or at population level (herd immunity).2
Direct effects of a vaccine include reducing the chances of a vaccinated individual becoming infected if exposed to the pathogen (ie, reduced susceptibility) and disease related outcomes despite acquiring infection.1 For some infectious diseases, the direct, protective effect of a vaccine against infection is the primary goal, such as with rabies, whereas for many the added downstream benefit of a reduction in symptoms if a vaccinated person becomes infected is an important secondary goal (eg, varicella). For other infectious diseases such as influenza and covid-19, the direct protective effect of reducing severe outcomes (admission to hospital or death) is the critical goal of vaccination, in addition to direct and indirect effects against infection.
When a vaccine reduces the chances of the vaccinated person passing on the pathogen if infected (ie, reduced infectiousness), the indirect effect can be to reduce risk of infection among both vaccinated and unvaccinated contacts.1
Finally, when the proportion of the susceptible population decreases because of vaccination (through the direct effect of reduced susceptibility), individuals who remain unvaccinated also experience a lower risk of becoming infected through herd immunity.1 Indeed, vaccinated people also experience an added benefit of lower risk of infection because of herd immunity. Herd immunity manifests at the population level and is a downstream consequence of the direct effects of a vaccine on reducing a vaccinated individual’s susceptibility to infection. The benefits of herd immunity are also amplified at the population level through indirect effects that reduce a vaccinated individual’s ability to pass infection on to others. Direct and indirect benefits can accrue to markedly reduce infection rates in a population, even to the point of eliminating or eradicating the disease, such as occurred with rubella and smallpox.
Studies of vaccine efficacy and effectiveness
Vaccine efficacy refers to how well a vaccine reduces outcomes in randomised clinical trials. The usual outcome of interest in such trials of vaccines is a reduction in acquisition of infection, symptomatic infection, or disease severity among vaccinated people. Owing to randomisation, individuals in the vaccinated and unvaccinated groups are considered “exchangeable,” or comparable, with respect to their risk of outcomes in the absence of vaccination, which means the risk of confounding is minimised. However, randomised controlled trials of vaccines may be infeasible (and sometimes unethical) during public health emergencies.
In 2022, the modified vaccinia Ankara-Bavarian Nordic (MVA-BN) vaccine was rapidly procured by and deployed in higher income countries to prevent mpox infections and reduce severity in these settings, with the recognition at the time that clinical evidence that a smallpox vaccine would confer protection against mpox was limited. Across higher income countries, the mpox outbreak disproportionately affected gay, bisexual, and other men who have sex with men in the context of sexual transmission, and thus regional pre-exposure vaccine programmes prioritised those most at risk of exposure to mpox virus.3 In the absence of randomised controlled trials and in the context of rapid implementation of vaccination programmes, observational studies were used to estimate how well the mpox vaccine worked in the real world (ie, vaccine effectiveness).4
Observational studies to measure vaccine effectiveness can examine direct and indirect outcomes. One increasingly used approach to estimate vaccine effectiveness involves leveraging existing or routinely collected clinical, laboratory, vaccination, and health administrative data. One of the limitations of using real world data is that outcomes depend on testing for the outcome—and in many observational studies the evaluation of effectiveness is narrowed to reduction in symptomatic or laboratory confirmed infection, or both.
Direct effect of MVA-BN vaccine against confirmed mpox infection
Using such a data platform in the Canadian province of Ontario, we estimated the effect of MVA-BN vaccine on reducing laboratory confirmed mpox infection among those who were vaccinated versus those who were unvaccinated (doi:10.1136/bmj-2023-078243).5 That is, we were interested in the direct, individual level effect of the vaccine among vaccinated people. The outcome of mpox infection was, by necessity, a laboratory confirmed diagnosis. Thus, the outcome did not capture those who received a clinical diagnosis of mpox without laboratory confirmation—a situation that was uncommon in higher income countries after the early period of the outbreaks but is common in high burden regions with limited diagnostic capacity.
The results of our study also do not necessarily explain how effective MVA-BN vaccine is at reducing symptomatic infection. Given that the criteria for laboratory testing was symptomatic infection, however, our estimate of vaccine effectiveness against laboratory confirmed infection was expected to be comparable to effectiveness against symptomatic infection. Our outcome measure also could not provide information on reduction in severity of disease; however, given the estimate for vaccine effectiveness of 58% (95% confidence interval 31% to 75%)5 against infection, even modest effectiveness against severity—conditional on becoming infected—would yield a large, direct reduction in severe infection among vaccinated people exposed to mpox virus. Finally, although our study was not designed to evaluate indirect effects or herd immunity, our finding suggests that the regional MVA-BN vaccination programme may have been a contributing factor in slowing mpox transmission in the region in 2022.6
Footnotes
Funding and competing interests available in the linked paper on bmj.com.
Provenance and peer review: Commissioned; not externally peer reviewed.