Incorporating dose effects in network meta-analysis
BMJ 2022; 376 doi: https://doi.org/10.1136/bmj-2021-067003 (Published 18 January 2022) Cite this as: BMJ 2022;376:e067003
- Jennifer A Watt, geriatrician1 2,
- Cinzia Del Giovane, statistician3,
- Dan Jackson, statistician4,
- Rebecca M Turner, statistician5,
- Andrea C Tricco, methodologist1 6,
- Dimitris Mavridis, statistician7,
- Sharon E Straus, geriatrician1 2 6,
- Areti-Angeliki Veroniki
, statistician1
- 1Knowledge Translation Program, Li Ka Shing Knowledge Institute, St Michael’s Hospital, Toronto, ON, Canada
- 2Division of Geriatric Medicine, Department of Medicine, University of Toronto, Toronto, ON, Canada
- 3Institute of Primary Health Care, University of Bern, Bern, Switzerland
- 4Statistical Innovation Group, AstraZeneca, Cambridge, UK
- 5MRC Clinical Trials Unit, University College London, London, UK
- 6Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada
- 7Department of Primary Education, School of Education, University of Ioannina, Ioannina, Greece
- Correspondence to: A A Veroniki areti-angeliki.veroniki{at}unityhealth.to
- Accepted 15 October 2021
In contrast to pairwise meta-analysis, which directly compares one treatment’s efficacy or safety to another based on head-to-head data, network meta-analysis (NMA) simultaneously compares and ranks multiple treatments that are either directly compared through head-to-head data, indirectly compared through a common treatment comparator, or both (that is, by a mixed treatment comparison composed of direct and indirect evidence).3 If a researcher wants to compare the efficacy or safety of multiple treatments, NMA can better answer this question than pairwise meta-analysis. The ability of NMA to simultaneously compare the efficacy and safety of multiple treatments has led to a sharp rise in the number of published NMAs and research to improve their methodological rigor.4
NMAs improve decision making by filling knowledge gaps where no head-to-head data exist when comparing treatments, but an absence of NMA results concerning treatment dose effects could limit their applicability and validity. For example, although it is helpful to know that donepezil, galantamine, and rivastigmine (drugs used to improve the symptoms of Alzheimer’s disease) are associated with an increased risk of nausea, clinicians could better support tailored decision making if they know which drug doses are associated with this risk.2 A lack of methodological guidance for researchers on how to incorporate treatment dose effects into systematic reviews with NMAs is contributing to this critical omission.
In this article, we aim to present three hierarchical NMA models that researchers can implement to incorporate dose effects into systematic reviews with NMA, even in the absence of previous knowledge of how to model a dose-response association; give practical guidance on how to conduct these analyses; provide empirical examples so that readers can appreciate the importance of modelling dose effects; describe considerations for evaluating the appropriateness of NMA models incorporating dose effects; discuss considerations in appraising the applicability and validity of systematic reviews with NMA incorporating dose effects; and highlight the challenges, limitations, and future research directions related to the selection of NMA models incorporating dose effects. Our empirical examples describe the dose effect association between atypical antipsychotic drug use and the risk of cerebrovascular events in people with dementia, and between cholinesterase inhibitor use and the risk of nausea or headache in people with Alzheimer’s disease, although the NMA models described could apply to examples in any medical discipline.12
Summary points
Systematic reviews with network meta-analysis (NMA) that ignore potential dose effects could limit the applicability and validity of review findings
Hierarchical random effects NMA models incorporating dose effects assume dose level consistency and that dose effects are equal, separate, or exchangeable; these NMA models do not make assumptions about the shape of dose-response associations
Although researchers should first consider clinical and pharmacological factors when selecting the most appropriate NMA model, statistical and methodological considerations are also important (eg, heterogeneity between studies or between doses, consistency across treatment and dose effects, and model fit)
Clinicians and other knowledge users should appraise the applicability and validity of NMA modelling assumptions, including explanations of the model selection process and biological plausibility for incorporating (or not incorporating) dose effects
Modifying hierarchical NMA models to incorporate dose effects
In the standard NMA model, consistency is assumed, random treatment effects are modelled, and effect estimates (eg, odds ratios, mean differences) are derived at the treatment level; dose effects are not explicitly modelled.3 In this article, we show how this hierarchical NMA model can be modified to incorporate dose effects.
Let us consider a hypothetical network of five treatments (a, b, c, d, and e) and 11 different doses indexed with tiT, where i=1, . . . ,11. In the left panel of figure 1, treatment a is the network reference node, which is a treatment with one or no dose (eg, placebo), and treatments b, c, d, and e represent other nodes; in the right panel, treatments are composed of doses. Here, we present three hierarchical random effects NMA models incorporating dose effects (see also supplement file 1), which vary according to the following factors:
Fictional example with network nodes representing treatments (treatment level network) and doses (dose level network). Thickness of solid lines is proportional to the number of studies included in the group comparison, and node size is proportional to the number of patients included in the underlying group. Dashed oval lines=doses that belong to the same parent treatment
If they assume consistency at the treatment level (that is, between direct and indirect comparisons)
The number of variance components
If they account for the relation between dose and parent treatment
Whether effect estimates are derived at the treatment level, dose level, or both (table 1).5 None of these NMA models assumes a parametric dose-response association (that is, using information about the mean and deviation from the mean to make assumptions about the population distribution).
Properties of hierarchical network meta-analysis models incorporating dose effects
Hierarchical random effects NMA dose effects models incorporating dose effects have three main sources of variation (table 1, fig 2): variation within studies, variation between studies, and variation between doses within treatments. The first level of variation is within studies (that is, the variability across study participants), which is modelled in a conventional way whereby each study has its own study specific baseline.3 The second level of variation is between studies: the variability in true effects across studies within each treatment dose comparison.6 In contrast to the standard NMA model, where variation between studies is modelled at the treatment level, hierarchical NMA dose effects models incorporate variation between studies at the dose level.3 In a random effects model, each study specific true effect size is part of a distribution of all true effect sizes and the variance of this distribution represents the variance between studies. The third level of variation is the variation between doses within treatments, which refers to the variability of dose effects within each treatment category, assuming that each dose corresponds to a specific treatment category. All three models incorporate variation within and between studies in the same way; however, only the exchangeable NMA dose effects model (model 3) incorporates variance components for all three potential sources of variation.
Graphical representation of sources of variance in dose effects models. Solid lines connecting nodes imply doses are directly compared in a randomised trial. t=treatment dose; t1, t2, t3=different treatment doses corresponding to doses 1, 2, and 3 in the entire network; white diamonds=summary dose effects; purple dashed lines=distribution within studies, and the width of the bell shaped distribution shows the variance within studies (variation within studies); solid curved lines=distributionbetween studies, and the width of the bell shaped distribution shows the variance within each comparison (variation between studies); black dashed oval=includesdifferent doses that belong to the same parent treatment; variation between doses within treatments=variance between treatment doses within treatments
Equal dose effects (model 1)
The simplest NMA model incorporates equal average dose effects (table 1, fig 3). This approach can only be considered for research questions targeted at assessing treatment effects, because it assumes that different doses of the same treatment are fixed and equally efficacious or safe within the same treatment group. This NMA model might include studies with data on multiple doses for the same treatment, but the dose effects are fixed and equal to the broader treatment effect. Data from study arms where the relative effects are assumed equal to zero contribute to the variance estimation between studies. An NMA model incorporating equal dose effects accounts for variation within studies and between studies, assumes consistency at the treatment and dose levels, and produces effect estimates (eg, log odds ratio) at the treatment level.
Graphical representation of networks according to how dose effects are incorporated into network meta-analysis models. Panels show networks incorporating equal, separate, and exchangeable dose effects. Thickness of solid lines is proportional to the number of studies included in the group comparison, and node size is proportional to the number of patients included in the underlying group. t=treatment dose; t1 to t11=different treatment doses corresponding to doses 1-11 in the entire network; a, b, c, d, and e=different treatments corresponding to each of the five treatments in the network; solid oval circles=dose effects are fixed and equal to the broader treatment effect; dashed oval circles=average dose effects are related and exchangeable within their parent treatment
Separate dose effects (model 2)
The second NMA model incorporates separate average dose effects (table 1, fig 3). It is appropriate for research questions assessing the effects of specific treatment doses, as it accounts for different dose effects. This NMA model assumes that average dose effects are unrelated with respect to their parent treatment and each other, and each node in the network is a separate treatment dose; therefore, the treatment dose association is not considered. The NMA model incorporating separate dose effects accounts for variation within studies and between studies, assumes consistency at the dose level only, and produces effect estimates at the dose level.
Exchangeable dose effects (model 3)
This third NMA model assumes that the average dose effects are related and exchangeable within their parent treatment (also known as exchangeable subnodes; table 1, fig 3).7 The model accounts for the association between treatment and dose, distinguishes between different treatment doses, and assumes that average dose effects within the same treatment come from a common distribution. It accounts for variation within studies, between studies, and between doses within treatments using variance components; assumes consistency at the dose level only; and produces effect estimates at both the treatment and dose levels.
Model 3 does not require additional assumptions about how to model the shape of the dose-response association (like models 1 and 2); accounts for the treatment dose association (like model 1); distinguishes between different treatment doses (like model 2); explicitly models variation within studies, between studies, and between doses within treatments using variance components; and produces effect estimates on both the treatment and dose levels. Therefore, this NMA model incorporating exchangeable dose effects is a preferred model for understanding different treatment doses if statistical and methodological considerations are valid (eg, dose level consistency and transitivity; box 1 and box 2). When the variance between doses is estimated as zero, this model simplifies to the NMA model incorporating equal dose effects (model 1).
Factors to consider when choosing network meta-analysis models incorporating treatment and dose effects
Anticipated clinical significance of treatment and dose effects (that is, network meta-analysis results should incorporate clinically relevant dose effects)
Heterogeneity between studies and between doses
Appropriateness of assuming transitivity and consistency at the treatment level, dose level, or both
Model fit and parsimony
Network geometry, connectedness (that is, avoidance of disconnected network components), and sparsity
Advantages to implementing hierarchical network meta-analysis models incorporating random dose effects within treatments (model 3)
Considers the association between treatment and dose
Does not make any parametric assumptions about potential dose-response associations
Facilitates borrowing of strength within treatment classes when different doses are available
Allows for the inclusion of studies comparing only multiple doses of the same treatment
Facilitates the simultaneous identification of the best treatment and dose
Can increase power compared to carrying out several independent subgroup analyses or extreme splitting approaches (that is, model 2)
Illustrative examples
We illustrate the NMA models mentioned previously with three empirical examples, which are presented below.12 For each example, we present the following:
Network plots
Transitivity tables
Model fit statistics (that is, deviance information criterion)
Estimates of heterogeneity between studies and between doses
Global (that is, design-by-treatment interaction model) and local (that is, loop specific approach) inconsistency estimates at the treatment and dose levels
Outcomes as medians with 95% credible intervals and 95% prediction intervals
Rankings according to surface under the cumulative ranking curve (SUCRA) values (100% indicates the best performing treatment and 0% indicates the worst).58910
We summarised SUCRA values for each outcome across models in a rank heat plot.11 We performed analyses in OpenBUGS: model fit and estimation methods are described in supplement file 2; OpenBUGS model code is available in supplement file 3; and all study data, transitivity tables, model fit statistics, heterogeneity estimates, inconsistency plots, treatment and dose level outcomes, and treatment and dose rankings can be found in supplement tables 1-12 and figure 1, figure 2, and figure 3.12
Atypical antipsychotic drugs
Dataset
Antipsychotic drugs are prescribed to people with dementia to treat neuropsychiatric symptoms (eg, aggression), but they are associated with potential harms in this patient population, including an increased risk of cerebrovascular events.113 Our example dataset is a subset of data describing the risk of cerebrovascular events associated with atypical antipsychotic use (that is, quetiapine, olanzapine, or risperidone) in people with dementia, which was published in a systematic review and NMA describing the comparative safety of drug interventions for treating neuropsychiatric symptoms in people with dementia (supplement table 1).1 Here, we include only those randomised trials that reported a target or average total daily treatment dose. We categorised treatment doses based on average total daily dose, where reported; otherwise, we categorised doses using target total daily dose. We categorised atypical doses of antipsychotic drugs according to ranges proposed by Maust et al: low dose (<125 mg/day), medium dose (125 to 200 mg/day), and high dose quetiapine (>200 mg/day); low dose (<5 mg/day), medium dose (5 to <7.5 mg/day), and high dose olanzapine (≥7.5 mg/day); and low dose (≤1 mg/day), medium dose (>1 to 2 mg/day), and high dose risperidone (>2 mg/day).14
Results—cerebrovascular events
We included 10 studies (3079 patients), four treatments, and seven treatment doses in our hierarchical NMA models incorporating treatment and dose effects (fig 4). We saw differences in dementia types and study duration across treatment and dose comparisons (supplemental tables 2 and 3). Low heterogeneity between studies was evident in the network, which did not change substantially across models (supplement tables 4A-D). Model fit was similar across models. We did not identify any global or local inconsistency at the treatment or dose levels (supplement figs 1A and 1B). These results suggest that researchers could implement models 1, 2, or 3, depending on their clinical or policy question.
Network diagrams depicting network connectedness of treatments and treatment doses (low, medium, high) for three illustrative examples (cerebrovascular events, nausea, and headache). Thickness of solid lines is proportional to the number of studies included in the group comparison, and node size is proportional to the number of patients included in the underlying group. Dashed oval lines=doses that belong to the same parent treatment
In model 1, olanzapine (odds ratio 3.18, 95% credible interval 1.12 to 9.52, 95% prediction interval 0.97 to 10.75) and risperidone (3.59, 1.71 to 8.03, 1.42 to 9.43) were associated with greater odds of cerebrovascular events than placebo. In models 2 and 3, medium dose olanzapine, low dose risperidone, and medium dose risperidone were associated with greater odds of cerebrovascular events than placebo (fig 5 and supplement tables 4A-C). With respect to treatment rankings (that is, SUCRA values), model 1 suggested that quetiapine was the safest and risperidone was the most harmful. With respect to treatment dose rankings, model 2 suggested that low dose olanzapine was the safest; low and medium dose risperidone were the most harmful. Model 3 suggested that low and medium dose quetiapine were the safest, whereas low and medium dose risperidone were the most harmful (fig 6 and supplement table 4D). Our results suggest that both low dose olanzapine and low and medium dose quetiapine are the safest treatment options for people with dementia because they are not associated with increased odds of cerebrovascular events.
Forest plot of odds ratios (95% credible intervals) describing the association between treatment doses of atypical antipsychotic drugs (olanzapine, quetiapine, and risperidone) and odds of cerebrovascular events compared to placebo. Network meta-analysis models incorporating separate or exchangeable dose effects were used to compare four treatments and seven treatment doses, including a placebo group
Rank heat plots for the outcomes of cerebrovascular events, nausea, and headache across treatment and treatment doses, based on use of three network meta-analysis models to compare treatments and treatment doses. Each model corresponds to a separate ring: outside circle=equal dose effects (network meta-analysis model 1); middle circle=separate dose effects (model 2); inner circle=exchangeable dose effects (model 3). Sectors are coloured according to surface under the cumulative ranking curve (SUCRA) values: red (0%; worst performing treatment), yellow (50%), and green (100%; best performing treatment)
Cholinesterase inhibitors
Datasets
Cholinesterase inhibitors (that is, donepezil, galantamine, and rivastigmine) are prescribed to people with dementia to slow cognitive decline. However, the drugs are associated with potential harms, including nausea and headache.2 Our example datasets are subsets of data describing the risk of nausea and headache associated with cholinesterase inhibitor use in people with Alzheimer’s disease, which were published in a systematic review and NMA describing the comparative effectiveness and safety of cognitive enhancers in people with Alzheimer’s disease (supplement tables 5 and 6).2 Here, we include only those randomised trials that reported a target or average total daily treatment dose. We categorised treatment doses based on the average total daily dose, where reported; otherwise, we categorised treatment doses based on the target total daily dose. We categorised cholinesterase inhibitor doses according to ranges proposed by Lee et al: low dose (≤5 mg/day) and high dose donepezil (>5 mg/day); low dose (<16 mg/day) and high dose galantamine (≥16 mg/day); and low dose (<6 mg/day) and high dose rivastigmine (≥6 mg/day).15
Results—nausea
We included 41 studies (10 604 patients), four treatments, and seven treatment doses in our hierarchical NMA models describing the association between cholinesterase inhibitor use and nausea (fig 4). Study and participant characteristics were similar across treatment and dose comparisons (supplement tables 7 and 8). Moderate heterogeneity between studies was evident in model 1 (variance between studies 0.20, 95% credible interval 0.06 to 0.49), which decreased substantially in models 2 and 3 (supplement tables 9A-C). Model 1 (deviance information criterion=157) fit the data better than model 2 (167) and model 3 (165).
Although no inconsistent network loops were evident at the treatment level, inconsistency was identified at the dose level for the loop involving placebo, low dose donepezil, and high dose galantamine (supplement figs 2A and 2B). Given the presence of one inconsistent network loop at the dose level, researchers could cautiously proceed with implementing models 1, 2, or 3; however, they could consider an alternative approach (box 3).3 The heterogeneity between studies was lower in models 2 and 3 than in model 1, which suggests that treatment dose explains part of this heterogeneity. If researchers proceed with implementing NMA models that assume consistency at the dose level because of important clinical considerations, they should implement model 2 or 3, depending on whether they are interested in dose effects only (model 2) or treatment and dose effects (model 3).
Alternative knowledge synthesis approaches if assuming consistency at the dose level in network meta-analysis models is inappropriate
Apply a model that assumes consistency at the treatment level only (as proposed by Dias et al3)
Incorporate random inconsistency effects in the dose effects model
Explore inconsistency and intransitivity through meta-regression or subgroup analyses
Apply pairwise meta-analysis models only
Narratively synthesise systematic review findings without performing meta-analysis
In model 1, donepezil (odds ratio 1.72, 95% credible interval 1.24 to 2.45, 95% prediction interval 0.65 to 4.79), galantamine (2.98, 2.05 to 4.31, 1.09 to 8.12), and rivastigmine (3.78, 2.61 to 5.59, 1.4 to 10.44) were associated with greater odds of nausea than placebo. In model 2, high dose rivastigmine was associated with greater odds of nausea than all other treatments; and high dose galantamine was associated with greater odds of nausea than high dose donepezil, low dose donepezil, and placebo. In model 3, high dose rivastigmine was associated with greater odds of nausea than all treatments except high dose galantamine; and high dose galantamine was associated with greater odds of nausea than high dose donepezil, low dose donepezil, and placebo (fig 7 and supplement tables 9A-C). With respect to treatment rankings, model 1 suggested that placebo was the safest and rivastigmine was the most harmful treatment. Models 2 and 3 suggested a dose-response across treatment doses (that is, high treatment doses had the least favourable treatment dose profiles; fig 6 and supplement table 9D). Our results suggest that high dose galantamine and high dose rivastigmine are associated with increased odds of nausea in people with Alzheimer’s disease, and that low doses rather than high doses of cholinesterase inhibitors are associated with more favourable nausea risk profiles.
Forest plot of odds ratios (95% credible intervals) describing the association between treatment doses of cholinesterase inhibitors (donepezil, galantamine, and rivastigmine) and the odds of nausea and headache compared to placebo. Network meta-analysis models incorporating separate or exchangeable dose effects used to compare four treatments and seven treatment doses, including a placebo group
Results—headache
We included 31 studies (8589 patients), four treatments, and seven treatment doses in our hierarchical NMA models describing the association between cholinesterase inhibitor use and headache (fig 4). Study and participant characteristics were similar across treatment comparisons, but we saw differences across dose comparisons with regards to study duration (supplement tables 10 and 11). Heterogeneity between studies was the highest in model 1 (variance between studies 0.28, 95% credible interval 0.07 to 0.76). Deviance information criteria across models were similar. We saw no evidence of inconsistency at the treatment or dose levels (supplement figs 3A and 3B). These findings suggest that researchers should implement models 2 or 3 because of the lower estimated heterogeneity between studies in these models than in model 1, depending on whether clinical interest is in deriving dose effects only (model 2) or both treatment and dose effects (model 3).
In model 1, only rivastigmine was associated with increased odds of headache compared to placebo (odds ratio 2.19, 95% credible interval 1.35 to 3.62, 95% prediction interval 0.65 to 7.57). In model 2, high dose rivastigmine was associated with increased odds of headache compared to placebo, high dose donepezil, and low dose rivastigmine. In model 3, only high dose rivastigmine was associated with increased odds of headache compared to placebo (fig 7 and supplement tables 12A-C). With respect to treatment ranking, model 1 suggested that placebo was the safest and rivastigmine was the most harmful treatment. Models 2 and 3 suggested a dose-response across treatment doses (that is, high treatment doses had the least favourable treatment dose profiles; fig 6 and supplement table 12D). Our results suggest that high dose rivastigmine is associated with increased odds of nausea in people with Alzheimer’s disease, and that low doses rather than high doses of cholinesterase inhibitors are associated with more favourable headache risk profiles.
Discussion
Clinical importance of modelling both treatment and dose effects
NMA models that reflect real life clinical experiences are important; if studies incorporate clinically relevant treatment doses, then researchers should use NMA models incorporating dose effects so that results are responsive to the needs of decision makers, unless methodological or statistical considerations exist that could jeopardise NMA conclusions (box 1). For this reason, the equal dose effects model (model 1) is only recommended when it is plausible to assume that any dose effects are small or absent because model 1 ignores possible differences in dose effects within treatments (box 2 and fig 3). Similar to model 3, model 2 incorporates both treatment and dose effects, but model 2 ignores potential treatment dose associations, derives only dose effects, and does not explicitly model between dose variation within treatments using variance components. Model 3 is highly appropriate for helping decision makers understand the comparative efficacy or safety of multiple treatments and different doses simultaneously. Hierarchical NMA models can also be extended to instances where describing the effects of treatment formulations (eg, oral, intravenous) and potential effect modifiers (that is, meta-regression) is important. Further, these NMA models could be modified to incorporate a parametric dose-response.
Our examples demonstrate the effects of both treatment and dose, which provide decision makers with important information beyond what was previously available in published medical literature.11316 Firstly, our results showed that use of the atypical antipsychotic drugs risperidone and medium dose olanzapine was associated with increased odds of cerebrovascular events, which could prompt clinicians to prescribe quetiapine or low dose olanzapine to avoid this adverse event. Secondly, we demonstrated a potential treatment and dose-response association between nausea and the use of several cholinesterase inhibitors—low dose donepezil was the best tolerated and high dose rivastigmine was the worst tolerated. However, decision makers need to cautiously interpret these findings because we detected local inconsistency in this NMA model. Lastly, if we had modelled only treatment effects, we would have assumed all doses of rivastigmine were associated with increased risk of headache; by incorporating dose effects, we found that this increased risk was associated with high dose rivastigmine only.
Dose effects as a source of heterogeneity
NMA models should reflect our real life clinical understanding of treatment doses: we assume that a treatment dose association exists (that is, doses of one treatment are more similar than are doses of another treatment) and how we model heterogeneity should reflect this understanding (box 1). Further, if the estimated variation between studies is sensitive to model choice, then reviewers can investigate with subgroup, sensitivity, or meta-regression analyses to understand if dose variability is an effect modifier or if participant characteristics vary by treatment dose (table 1). For example, in our empirical examples of cholinesterase inhibitors, the equal dose effects model (model 1) increased estimated heterogeneity between studies compared to the models incorporating separate dose effects (model 2) and exchangeable dose effects (model 3).
Appropriateness of assuming transitivity and consistency at the treatment level, dose level, or both
Transitivity implies that effect modifiers are balanced across NMA treatment and dose comparisons; consistency is the statistical quantification of transitivity. Researchers should evaluate these assumptions at each level that they are assumed (that is, transitivity and consistency assumptions must be assessed on both the treatment and dose levels if researchers apply model 1). In addition to intransitivity or inconsistency related to dose effects, inconsistency might also be due to an imbalance in the distribution of other effect modifiers (eg, participant age, sex, dementia severity). We did not identify any global or local inconsistency at the treatment level in our examples. At the dose level, we identified one inconsistent network loop in our example where we described the association between cholinesterase inhibitor use and risk of nausea; dose effects estimated from direct evidence differed significantly from those estimated from indirect evidence in the closed network loop incorporating placebo, low dose donepezil, and high dose galantamine.5 Where inconsistency or intransitivity is identified at the dose level, applying a model assuming consistency at the dose level might not be appropriate, and researchers should consider alternative approaches (box 3).317 Researchers should explore a number of factors (eg, variance between studies, transitivity, consistency, model fit statistics) before choosing between models (box 1). Fitting multiple models could improve understanding of the dataset and interpretation of results. Readers and peer reviewers of manuscripts reporting NMAs incorporating treatment and dose effects should also consider these factors when appraising the applicability and validity of systematic reviews with NMA (box 4).
Factors to consider when appraising the applicability and validity of systematic reviews with network meta-analysis (NMA) that incorporate treatment or dose effects, or both
Is the biological plausibility of incorporating dose effects explained?
If authors decide to incorporate dose effects, have they included all clinically relevant treatment doses?
If authors chose an NMA model incorporating dose effects, do they provide a valid rationale for their model selection process?
If NMA models incorporating dose effects assume a dose-response association, have authors justified how they chose to model this association?
If dose effects are not incorporated in NMA models, have authors explained why they made this decision (eg, network sparsity, dose level inconsistency, poor model fit, no biological plausibility, not relevant to the research question)?
Alternative approaches for incorporating dose effects
Alternative approaches to modelling dose effects in NMAs have been suggested.7181920 Del Giovane et al proposed several other hierarchical NMA models incorporating dose effects.7 Similar to model 3, reviewers could apply a NMA model incorporating random dose effects without assuming consistency at the dose level; however, this model can only be implemented if there are no multi-arm studies.7 Del Giovane et al also proposed a random walk process (that is, adjacent treatment doses could be modelled as more similar than non-adjacent doses) or it could be assumed that a monotonic dose-response association exists (eg, higher doses are likely to be more beneficial for clinical outcomes).7 These alternative hierarchical NMA models incorporating dose effects require that researchers make additional modelling assumptions, which should be carefully considered a priori by a multidisciplinary team (eg, content experts, methodologists, and statisticians).
Owen et al proposed a hierarchical NMA model that assumed a monotonic but non-parametric dose-response between nodes representing different doses of the same treatment.20 Owen et al implemented ordering constraints—that is, assumed that higher doses would be associated with the same or greater clinical benefit.20 Thorlund et al implemented a network meta-regression model that assumed a linear dose-response on the log odds scale and incorporated a three level categorical covariate for doses at half each drug’s common dose, each drug’s common dose, or double each drug’s common dose.19 In this model, assumptions must be made about each drug’s common dose, which can vary by study population. Mawdsley et al proposed a model based NMA framework that facilitates estimation and prediction of dose effects for multiple treatments within a drug class across a range of doses (including those for which study data are not available), using plausible physiological dose-response models.18
Challenges and limitations of applying NMA models incorporating treatment and dose effects
Use of NMA models incorporating dose effects has challenges and potential limitations. Firstly, studies that do not report treatment dosing information cannot be included in NMA models incorporating treatment and dose effects. Secondly, performing NMAs that assume equal average dose effects (model 1) could increase precision of treatment effects, but with potential trade-offs:
Greater heterogeneity and inconsistency if clinically meaningful dose effects are not included in the NMA model
NMA outputs that might be less meaningful for decision makers, especially if dose effects are believed to be clinically important.
In contrast, the splitting of treatment nodes into smaller, dose based, subnodes could reduce precision in effect estimates because there are fewer studies informing each NMA dose comparison (model 2). However, heterogeneity and inconsistency could decrease because the effects of dose on heterogeneity are explicitly modelled, and NMA outputs could be more meaningful for decision makers.
Thirdly, the splitting of nodes to incorporate dose effects might create treatment doses with zero events or disconnected networks. Fourthly, decisions about how to model dose-response associations in NMA models can be complicated, which is why we present three NMA models incorporating dose effects that do not require previous knowledge of this dose-response association. However, if researchers have confidence in how to model the association for treatments under study then alternative models can be considered, as proposed by Del Giovane et al and others.71820
In addition, given that studies reporting dose effects might have more than two arms and comparison adjusted funnel plots assume independence between effect estimates in multi-arm studies, researchers assessing for publication bias can instead implement a selection model (eg, Copas model) and present funnel plots for each direct treatment comparison.2122 Most direct treatment comparisons in our NMA models were informed by fewer than 10 studies so we could not evaluate for publication bias. Lastly, we implemented NMA models in a Bayesian framework, which might be less familiar to some researchers, but NMA models incorporating dose effects could alternatively be implemented in a frequentist framework. A Bayesian framework offers several advantages over a frequentist framework, including modelling flexibility, a simpler way to derive ranking statistics associated with treatment and dose effects, the ability to implement informative priors to estimate variance between studies, and a more intuitive interpretation of results for decision makers.
Conclusion
Having the ability to incorporate both treatment and dose effects is important for researchers aiming to produce relevant and clinically meaningful NMA results for decision makers. However, implementing NMA models incorporating treatment and dose effects is complex and requires the skills of a multidisciplinary team (eg, clinicians, methodologists, and statisticians). As we have highlighted, clinical and pharmacological considerations should be considered first, but statistical and methodological considerations are also important. Further, different approaches and decisions about network structure could generate important variations in results so that, when possible, decisions concerning NMA model assumptions should be made a priori. Future research to guide selection of NMA models incorporating dose effects will be critical for developing a consensus based approach and advancing knowledge synthesis methods incorporating NMA.
Ethics statements
Ethical approval
Not applicable.
Data availability statement
The full dataset and statistical code are available in the supplement material.
Footnotes
Contributors: JAW contributed to the study design, prepared datasets, completed analyses, drafted the initial manuscript, and integrated coauthor feedback. ACT and SES contributed to the study design and provided manuscript feedback. AAV conceived the study idea, developed model codes, completed analyses, drafted the initial manuscript, and provided manuscript feedback. All study authors contributed to the conception and design of this study. JAW and AAV conducted data analyses and drafted the first version of the manuscript. All authors contributed to the manuscript’s revision and interpretation of findings. JAW is the guarantor of this article. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.
Funding: This study had no funding. ACT is supported by a tier 2 Canada research chair in knowledge synthesis. SES is supported by a tier 1 Canada research chair in knowledge translation. RT is supported by the UK Medical Research Council (grant No MC_UU_12023/24).
Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/disclosure-of-interest/ and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; DJ is employed by AstraZeneca; no other relationships or activities that could appear to have influenced the submitted work.
JAW affirms that this manuscript is an honest, accurate, and transparent account of the study being reported and that no important aspects of the study have been omitted.
Dissemination to participants and related patient and public communities: We will disseminate our results to relevant knowledge user groups (eg, patients, care givers, healthcare managers, and clinicians).
Provenance and peer review: Not commissioned; externally peer reviewed.