Reporting of cluster randomised crossover trials: extension of the CONSORT 2010 statement with explanation and elaboration

Explanation

A rationale for the choice of trial design should be provided. For a CRXO trial, this rationale should include justification of the cluster and crossover aspects of the design.13 Justifying cluster randomisation is important because this will likely increase the required number of participants compared with an individually randomised trial. Thus, more participants will likely receive an intervention of unknown effectiveness than if individual randomisation were used. Furthermore, cluster randomisation could increase the potential for bias arising from selective identification and recruitment of participants. The crossover element might exacerbate this risk in cross sectional designs, because participants in the second period are, by definition, recruited after randomisation and it might be harder to ensure that those responsible for recruitment in the second period are unaware of the cluster’s assigned treatment. The crossover element might lead to an increased risk of cluster withdrawal owing to additional burden of participating in multiple periods. Hence, justifying the need for the crossover element is also important.

Possible justifications for adopting a cluster randomised design include the ability to evaluate interventions directed at clusters, operational and logistical benefits, and the need to avoid contamination within clusters (when interventions are delivered at the individual level).11 Possible justifications for using the crossover include infeasibility of a parallel design due to the availability of too few clusters to achieve the desired power29; reduction of the impact of chance imbalances in cluster characteristics; and increased statistical efficiency (with associated benefits such as minimising the number of participants receiving the interventions, and reducing the cost).

A key requirement of the crossover design is that the effect of an intervention given in one period does not carry over into the next period.30 In a CRXO trial, the carryover can occur at both the cluster and individual levels.31 At the cluster level, the treatment delivered in one period can continue to be delivered in subsequent periods. At the individual level in cross sectional designs, participants who have treatment for a long period (eg, those receiving care in an intensive care unit) might unintentionally be given the other treatment if they are still in the cluster when it crosses over. In open or closed cohort designs where participants are followed over time, carryover could arise in the same ways as in individually randomised crossover designs (ie, the treatment itself could persist (eg, physical persistence of a drug) or the effects of the treatment might persist (eg, a drug has a curative effect)).81530 Therefore, methods for managing potential carryover (eg, the use of washout periods, along with justification for the duration; different participants in each period; and blinding of trialists involved in the delivery of the intervention), or justification for why carryover at both the cluster and individual level is expected to be negligible, should be provided. In a 2×2 crossover design, carryover is particularly problematic because it cannot be distinguished from a treatment by period interaction.30

In CRXO trials where period effects are expected, any design approaches used to limit the possible impact of such period effects should be noted. An example of a period effect is the seasonal variation in outcomes that might be expected when the trial spans a year and the outcome (eg, incidence of influenza) varies by season. A learning effect is another form of period effect that might arise when fidelity to each intervention improves similarly in the second period. The use of balanced designs and dual pair sequences (eg, ABB and its dual BAA) could limit the potential for period effects to bias the estimation of treatment effects.11

Providing specific details of the CRXO design is important to allow assessment of the potential for bias, and whether the sample size and analysis methods were appropriate. Where possible, justification for the chosen design elements should be provided (eg, providing a rationale for the chosen composition of sequences). A diagram can efficiently and clearly communicate the design details, particularly for complex designs (more than two periods or treatment conditions or both) or simple designs with staggered commencement of clusters. Key details to depict for each sequence include the commencement date and duration of each treatment condition and washout period, and the number of clusters allocated (example 3 of item 3a.2). Such a diagram will clearly signal any imbalance in the treatments evaluated in each period, and therefore the need to control for period effects.

Example of item 3a.1

Example 1 (rationale for cluster randomisation and crossover)—“Cluster-randomisation with hospitals as clusters was used to overcome confounding by indication and to prevent contamination between isolation strategies. Crossover of strategies at the hospital level was aimed at reducing between-hospital variability and, thus, increasing statistical efficiency.” 32

Examples of item 3a.2

Example 2 (description of cross sectional design)—“Hospitals were randomized 1:1 . . . to UCM [umbilical cord milking] or ECC [early cord clamping] in Period 1 (5 hospitals per treatment group from January 2019 to January 2020) until half of the required enrollment was reached (N=600 consented subjects); they were then crossed over to the other intervention during Period 2 (February 2020 to May 2021) for the remaining half of consented subjects (N=600). A 1- to 2-month washout period occurred after Period 1.”28

Example 3 (description of closed cohort design; number of schools allocated to each sequence shown in figure 2 of Makris et al33)—“The trial was conducted in six primary schools with two periods (40-days organic diet vs. 40-days of conventional diet) . . . A washout period was not required as it was intrinsically included in the two periods, since the first urine sample [for measurement of the primary outcome] of the second period was collected about 12 days after the beginning of the second period. Moreover, the pesticide half-lives are short (half-lives ranging 6.4-16.5 hours for pyrethroids and 5-33 hours for neonicotinoids), so no carryover effect was expected.”33

Example 4 (diagram of realised design from MedBridge trial34)—see figure 3.

Fig 3

Diagram of the realised design from the MedBridge trial.34 AIM=acute internal medicine; ASN=acute stroke and neurology; DN=diabetes and nephrology; IM=internal medicine. Adapted from reference 28 with permission

• Download figure
• Open in new tab
• Download powerpoint

Explanation

Changes from the planned methods might need to be made after the trial has commenced. Important changes should be fully reported to enable readers to appropriately interpret the results. For example, changes in a CRXO design might arise from delays in recruitment of either clusters or participants within clusters, which could lead to unintended staggered commencement of the trial in different clusters, or differences in period lengths. Such changes have the potential to bias estimates of treatment effect, and therefore are important to describe.

Example of item 3b

Example (deviation from planned dates)—“The study had 1 major deviation. Originally, the initial phase was to run from November 2017 to February 2018, and the crossover phase from March to June 2018. However, because of logistics issues faced by all clinics with obtaining seasonal influenza vaccine supplies ahead of the midyear season, commencement of the crossover phase was delayed by 1 month. We retained the study design of two 4-month phases and ran the initial and crossover phases from November 2017 to February 2018 and from April to July 2018, respectively.”35

Explanation

A complete description of the intervention conditions is necessary for their future implementation.38 The TIDieR (template for intervention description and replication) checklist provides an extension of the CONSORT statement (item 5)3 for describing interventions.38 Complete description also allows assessment of the potential for carryover effects (table 1). Unlike other cluster designs (parallel, stepped wedge), the CRXO design is not suitable for evaluating treatments where their effects persist, such as educational interventions targeted at changing clinicians’ practice.30

For cluster trials, reporting whether the treatments are delivered at the cluster level, the individual participant level, or both, is important. This information allows assessment of whether individuals can avoid treatments (which is important for determining the coverage of the treatment) and has implications for the identification of research participants and requirements for informed consent (items 10c and 26).

In CRXO trials, treatments are commonly delivered at the individual participant level (eg, daily bathing of critically ill children in paediatric intensive care units with chlorhexidine gluconate compared with standard bathing practices) because these treatments can be withdrawn.8 Treatments delivered at the level of the cluster can also be evaluated using the CRXO design, as long as the intervention can be withdrawn (eg, enhanced cleaning of hand contact surfaces within intensive care wards compared with standard cleaning, where the active treatment component of extra hand gel on wards can be removed).

Examples of item 5

Example 1 (description of the treatment conditions; level of delivery to the individual)—“In each group, the intervention was applied at the patient level.

“For patients included during experimental periods, hospital pharmacists performed the medication reconciliation at discharge . . . Hospital pharmacists completed a short form documenting the reason for hospitalisation, home medication modifications, new medication and laboratory results necessary for community pharmacists to understand and/or accept the prescription . . . They also checked the discharge prescriptions (drug added and/or omitted, different dosage, route or duration of treatment) and, if needed, made an intervention on the physician’s prescription according to SFPC [French Society of Clinical Pharmacy] standards [standards provided in a figure] to change the prescription. Then they explained to the patient the drug initiated and the modifications to the home medication. They phoned the patient’s community pharmacist to explain the patient's inclusion in the study, the discharge time, and the modifications in treatment. They also sent the prescription sheet to the community pharmacist before patient discharge. The patient or care giver then visited the community pharmacist as usual.

“For the control group, patients received the usual care already implemented both at the hospital (classical drug dispensation by staff pharmacists) and by their community pharmacist (drug dispensation according to the prescription sheet written by the hospital physician in addition to the general practitioner’s sheet [if present]).” 39

Example 2 (level of delivery to the cluster)—“The study fluid interventions were implemented at the hospital level using a hospital policy or strategy, with the aim to answer our study question at the hospital level.” 37

Explanation

Outcomes should be completely defined, such that the resulting treatment effect estimates can be interpreted and used in other research (eg, systematic reviews). Here, we focus on issues particular to CRXO trials, and refer readers to the CONSORT 2010 statement and the CONSORT-Outcomes extension for further discussion of aspects common to all designs and examples.340

In cluster randomised trials, outcomes in each period can be measured at the level of the individual or at the level of the cluster. The second option could reflect a true cluster level outcome or arise because only aggregated individual level data are available (eg, the number of hospital associated infections per period might be available only in summary form).12 The level of the outcome measure has implications for the method of analysis (item 12a).

The timing of outcome assessments should be reported, including whether the outcomes were measured at discrete points in time common to all participants (eg, via a survey at the end of each period), or at a time point specific to each participant (eg, number of adverse events per patient until the time of their hospital discharge). Reporting of the timing of follow-up facilitates assessment of whether participants had similar (or variable) duration of exposure to the treatment conditions (table 1), and whether there was the possibility of contamination of observations collected in the second (or later) periods. Possible contamination might arise in cross sectional designs where the treatment is delivered at the level of the cluster.

In CRXO trials, data might be sought from routinely collected sources (eg, registries, hospital medical records) or purposively collected sources (eg questionnaires, patient diaries). Reporting the methods for data collection is important for allowing readers to assess whether the methods are appropriate for measuring the intended outcomes and facilitates an assessment of the potential for bias arising from measurement of the outcomes.41

Examples of item 6a

Example 1 (description of outcomes; primacy; outcome measured at level of individual; time point specific to each participant)—“Primary outcome and timing: All the outcomes pertain to the individual participant. The primary outcome was admission to the NICU [neonatal intensive care unit] in the first 24 hours of life for predefined criteria as follows: respiratory distress (tachypnea, grunting, or retractions), bradycardia or tachycardia, hypotonia, lethargy or difficulty arousing, hypertonia or irritability, poor feeding or emesis, hypoglycemia, oxygen desaturations or cyanosis, need for oxygen, apnea, seizures or seizure-like activity, hyperbilirubinemia, and/or temperature instability.

“Secondary outcomes and timing: The predefined secondary safety and efficacy outcomes included therapeutic hypothermia, volume expanders, phototherapy, haemoglobin at 24 hours of life, and peak serum bilirubin.”28

Example 2 (outcome measured at level of cluster; use of administrative data)—“Primary feasibility outcome: Adherence to the fluid protocol: Adherence to the study fluid was measured not at the individual patient-level, but according to the aggregate use of the study fluid throughout the hospitals (all hospital wards, monitored units, and departments) using the hospital inventory system; monitoring fluid exposure or adherence according to individual patients was not feasible due to the sheer number of hospital admissions . . .

“Data collection: All follow-up and collection of data for enrolled patients at the participating hospitals were captured through health administrative data that are housed at the Institute for Clinical Evaluative Sciences. There were no individual patient level data collected by research coordinators in the participating hospitals.” 37

Example 3 (description of outcomes; primacy; outcome measured at level of individual (implicit); time point specific to each participant; data collection methods)—“The primary outcome was symptomatic VTE [venous thromboembolism] within 90 days of surgery . . .

“Data source and timing: Patients received a web link via email or text message to complete online data collection at 90 days postoperatively . . . Data collection forms were reviewed by registry staff and all patients who responded “yes” to having had a venous thromboembolism, a secondary operation within 90 days or 6 months, a major bleeding event within 90 days, or a joint-related readmission within 90 days had this outcome verified through written documentation from treating physicians. These results were verified by the trial outcome verification committee. For the primary outcome, the date and type of VTE (below- or above-knee deep venous thrombosis [DVT] or pulmonary embolism) and side of DVT (left or right) were recorded. Mortality data were collected through linkage between the registry and the National Death Index.” 42

Explanation

Despite improvements over time, most randomised trials remain insufficiently powered to detect the small effects that are generally observed.43 Reporting the sample size calculation in sufficient detail is important to ensure reproducibility and for allowing assessment of the plausibility of the assumptions, and identification of any scientific or ethical concerns arising.19

The method used to determine the sample size should be described, specifically noting how the calculation accounts for both the cluster randomisation and the multiple period aspects of the design (seen in examples elsewhere214445). For CRXO trials, as in any cluster randomised trial, it is important to account for the intracluster correlation coefficient; however, in a CRXO trial, a single intracluster correlation coefficient might not be adequate because the strength of the intracluster correlation could depend on the separation in time of the observations. The recommended minimum items to report are given in table 4. The details to provide depend partly on the particulars of the design (eg, whether the design is cross sectional or cohort).

Justification for the choice of parameter values such as intracluster correlation coefficients should be provided (eg, based on estimates reported in a repository of correlations or extrapolation from known determinants of correlations such as whether the outcome is a process or clinical outcome).4647 Often sample size parameters (eg, intracluster correlation coefficients, cluster autocorrelation coefficient) are based on estimates with large uncertainty. Therefore, the protocol should include an indication of the sensitivity of the sample size, or power, to the assumed parameter values, and should be available with the trial report.19

In some CRXO trials, the sample size might be fixed by constraints on the number of clusters, participants, or periods. In this circumstance, the effect size that the trial was powered to detect should be reported. If no power calculation was performed, this should be reported. Retrospective power calculations based on the results of the trial are of little merit and are discouraged.48

Example of item 7a

PEPTIC trial (some core items are described elsewhere in the publication: two periods; two sequences; 1:1 allocation ratio)—“There is no established consensus regarding what constitutes a minimal clinically important difference in in-hospital mortality for critically ill adults. This . . . makes it challenging to establish what difference in mortality one should power an ICU study to detect . . . Accordingly, we sought to conduct the largest trial that we considered feasible in order to provide the most precise estimates of the likely mortality treatment effect possible with PPIs [proton pump inhibitors] vs. H2RBs [histamine-2 receptor blockers]. In the study protocol the sample size was calculated [reference to sample size method] to have 80% power at a 5% significance level to detect a 2.4% absolute reduction in in-hospital mortality from a baseline mortality of 15%, which corresponds to a relative risk reduction of 16%. This calculation assumed a mean cluster-period size of 310 patients with coefficient of variation of 0.50, a within-cluster-within-period correlation of 0.035, and a within-cluster-between-period correlation of 0.025, yielding a cluster autocorrelation of 0.71.” (Supplement 2 of PEPTIC trial.9)

Explanation

Interim analyses of outcomes are undertaken to inform the decision of whether a trial should be stopped early if there is substantial evidence of harm, effectiveness, or futility. Authors should report the dates or times at which they were carried out, what triggered them, whether these were planned or ad hoc, and the statistical methods used (including any formal stopping rules). When stopping rules or guidelines are used, it is important to report whether these were planned before commencement of the trial and before assessment of any interim analysis results, or some time thereafter.

For CRXO trials, little methodological guidance exists on how to do an interim analysis of outcomes, and interim analyses are challenging owing to the complexity of this design. For example, whether commencement of clusters is concurrent or staggered will affect the type of interim analysis. If commencement of clusters is concurrent, in a two period design when the analysis is undertaken at the end of the first period, the design would reduce to a parallel cluster randomised trial, necessitating a different analysis from the planned final analysis. If commencement of clusters is staggered, some clusters might have completed both treatments, others only one, and some a mix of partially completed treatments. Any interim analyses should address these issues, and the properties of any stopping rules should be determined, with these details reported. As with any trial, the planned interim analyses (where a decision is to be made about continuation of the trial) should be allowed for in power calculations to control the overall type I error rate.12

Other important reasons for considering whether to stop a trial arising from the trial conduct itself include: the inability to recruit clusters or clusters unable to recruit participants as planned, and poor quality of the primary outcome. Reporting the date when the trial was stopped, along with the reasons for stopping, is recommended.

Example of item 7b

“An interim analysis was not initially planned, as both treatments are considered standard practice for venous thromboembolism prophylaxis in Australia and the trial is investigating an adverse event as the primary outcome. However, due to concerns of an increased adverse event rate (symptomatic thromboembolism and death) in one of the prophylaxis groups, a Data Safety Monitoring Board (DSMB) was convened 1 year into patient recruitment . . . The DSMB were advised by the Trial Management Committee (TMC) to conduct an interim analysis. In conjunction with the DSMB (prior to the interim analysis), the TMC applied the Haybittle-Peto stopping rule of a two sided significance of 0.001 for the primary outcome in population 1*.

“This stopping rule was chosen as it does not require adjustment of the significance threshold for the final analysis and allows further interim analyses using the same threshold (if required). After the first interim analysis (in September 2020), the DSMB recommended continuing the trial and performing a second interim analysis in November 2020 . . .

“Interim analyses were conducted for thromboembolism and mortality within 90 days for population 1*. To account for unequal cluster sizes, incomplete crossover or clusters which had not yet crossed over, a composite analysis was designed. For clusters which had crossed over, including with partial completion of the second period, the cluster weighted estimator intended for the primary outcome was used. Harmonic mean weighting when there are unequal cluster sizes has been shown to improve precision and 95% confidence interval coverage compared with unweighted or inverse variance estimates [reference provided for methods]. Clusters which had not crossed over were analysed using the cluster period summaries, weighted by cluster size, in a parallel design approach, i.e., as if it were a cluster randomised trial without crossover. Estimates for the two approaches were combined using inverse variance weights to provide a final estimate. Confidence intervals were constructed using the Haybittle-Peto boundary of 0.001.” (Supplement 2 of Sidhu et al42; *Population 1=all registered patients undergoing primary total hip arthroplasty or total knee arthroplasty for a diagnosis of osteoarthritis.)

Explanation

In a CRXO trial, clusters are randomly allocated to a sequence of treatments, where the sequence defines the order in which each cluster will receive the treatments. For example, a two treatment, two period design might have two sequences: treatment A followed by treatment B (sequence 1), and treatment B followed by treatment A (sequence 2). The allocation schedule dictates the random allocation of clusters to either sequence 1 or 2. Note that our use of the term “sequence” differs from its use in the CONSORT statement3 and the cluster CONSORT extension,5 where for these parallel group designs, “sequence” refers to the randomly generated list of allocations to a single treatment (eg, treatment A or treatment B). In this CRXO statement, the term “schedule” is instead used to refer to this randomly generated list of allocations to sequences of treatments (eg, AB, or BA; table 5 presents an example).

The set of candidate sequences and the method used to generate the random allocation of clusters to these sequences should be sufficiently described so that an assessment can be made as to its adequacy. Use of general terms such as “random,” “randomisation,” and “random allocation” are insufficient. Details of the method are required (eg, random numbers generated by computer).

Example of item 8a

“Each hospital unit is randomised to perform medical reconciliation or usual care for the first 14-day period and is crossed over to the other group for a second 14-day period.” (Figure 1 of Pourrat et al.39)

“The randomisation sequence was generated . . . using a computerised process.”39

Explanation

In a CRXO trial, randomisation of clusters to the sequences of treatments is often done at a single point in time before the trial commences. In this circumstance, it might be possible to balance (restrict) the randomisation such that an equal number of clusters are allocated to each of the sequences of treatments, which can be achieved using a single block balanced across sequences for the entire study. For example, in a design of two periods, two treatments, and two sequences (AB or BA) with 20 clusters, a single block of size 20 balanced across the two sequences would allocate 10 clusters to AB and 10 to BA.

Stratification in CRXO trials might also be used for clusters with distinct characteristics (eg, different case mix, different hospital protocols for the delivery of care) that are related to the trial outcomes, or where commencement of clusters is staggered in time and there is potential for secular or seasonal trends. In stratified designs, balanced allocation of clusters to sequences are generated independently within each stratum. Other methods of restricted randomisation that might be used include covariate constrained allocation,49 minimisation,50 and matching.51

When reporting the type of randomisation, investigators should indicate whether clusters were allocated at one time, in batches or sequentially, and whether simple or restricted randomisation (eg, permuted blocks) was used. If permuted block randomisation was used, details should be provided on how the blocks were generated, the block size(s), and whether the block size was fixed or randomly varied. If stratification was used, researchers should provide details about the stratification variables and the cut-off values for categorisation.3 If another method of restricted randomisation was used, the method should be described, along with the variables used to restrict the randomisation.

Example of item 8b

Example 1 (restricted randomisation using stratification; sequential allocation)—“The ICUs [intensive care units] were consecutively randomized in a 1:1 ratio using computer-generated randomization with random block sizes of 2, 4, and 6 and stratified by number of ICU beds (≤10 or >10).”52

Consecutive randomisation was “as per the date of approval by the institutional review board” (supplement 1 of Rosa et al52).

Example 2 (restricted randomisation using stratification; batch allocation)—“Participating ICUs [intensive care units] were randomized to the order of treatments in a 1:1 ratio when ethics and regulatory approvals were in place at each study site. Randomization was performed using computer-generated random numbers and was stratified by region and time period with a minimum of 4 ICUs randomized at a time.”9

Example 3 (simple randomisation)—“Schools were randomized a priori to two groups that differed in the sequence of the treatments; organic diet followed by conventional diet (Group 1) or conventional diet followed by organic diet (Group 2).”33

“Schools were randomly assigned following simple randomization procedures of no restriction or matching.” (Supplement 5 of Makris et al.33)

Explanation

The benefits of randomisation are only realised when randomisation is properly implemented. In individually randomised trials, this process requires generation of a random allocation schedule, and concealment of upcoming assignments from the participants and from those responsible for recruiting the participants, until after recruitment (known as allocation concealment).53 Analogously, in cluster trials, where randomisation occurs at the level of the cluster, allocation concealment involves concealment of upcoming assignments from the clusters (ie, so-called gatekeepers or guardians) and from those responsible for recruiting the clusters. The randomisation process in CRXO trials might involve randomising clusters sequentially, in batches, or all at once.54 The strategies implemented to conceal the allocation will differ depending on the randomisation process, and should be completely described. As with other cluster randomised trials, allocation concealment is likely to be preserved when randomisation occurs at a single point in time.54

In cluster randomised trials, participants might be recruited (or identified) after the clusters have been allocated. This scenario can lead to recruitment or identification bias arising from selective recruitment (or identification) of participants across treatment conditions when those responsible for the recruitment are aware of the cluster’s allocation in a particular period. In this statement, the process for recruitment of individual level participants is considered separately (item 10b) to the randomisation process.

Example of item 9

Example 1—“Crossover and randomization took place at cluster (ward) level within each hospital . . . The computer-generated codes were held by the statistician to assure allocation concealment until the moment of randomization.” (Supplement 3 of Kempen et al.34)

Example 2—“Use of sequentially numbered sealed opaque envelopes enabled concealment of the allocation sequence at the hospital level until the last hospital was randomised.”32

Explanation

Consistent with the CONSORT extension for other clustered designs,512 it is important that all steps in the implementation of the randomisation and recruitment processes for both clusters and individual level participants are described. Reporting of these processes allows readers to assess the potential for bias. Details of the allocation and enrolment process of the clusters are described in item 10a, with the corresponding detail provided for participants in item 10b. The consent process for participants is also dealt with under implementation of the randomisation methods (item 10c) because this component is integral to enrolment, with potential implications for bias (eg, when consent procedures differ across treatment conditions that might lead to different types of participants being recruited). Use of the timeline cluster tool, which depicts the time sequence of trial processes (including the randomisation process) and the blinding status of participants and trial staff at each stage, allows readers to easily assess threats to the internal validity of the trial.55

Explanation

Reporting who generated the random allocation schedule, who enrolled the clusters, and who assigned the clusters to the sequences of treatments is important for allowing assessment of the potential for bias.54 A guiding principle to achieve adequate allocation concealment involves separation between the person who generates the allocation schedule and those who enrol and assign the clusters to the sequences of treatments. This separation can be achieved by having someone (preferably a statistician) independent of the trial generate the allocation schedule. When this person is unaware of the identity of the cluster (eg, where they receive a non-identifying unique code for each cluster), they might also assign the clusters to the allocation schedule (ie, equivalently maintaining separation). Failure to separate the above processes risks potential subversion of the randomisation schedule and potential biased estimates of treatment effect.

Example of item 10a

Example 1 (who generated the random allocation schedule, who enrolled clusters)—“Potentially eligible health services were identified in consultation with Departments of Health and Aboriginal Community Controlled Health Organisations and enrolled by study coordinators (AT, BH, and SGB) . . . The trial statistician (HW) did the randomisation . . . Participating health services were told their group allocation once participation agreements were signed.”56

Example 2 (who generated the random allocation schedule, who assigned clusters to the sequences of treatments)—“To remove the potential for allocation bias, one statistician generated the allocation codes and another randomly permuted clinic names within strata. A third person matched the allocation codes with clinic names to reveal the allocations.” 57 [Note that the role of the third person should be specified.]

Explanation

As with other cluster randomised designs,512 there are different sources from which the individual participant data can be obtained (eg, directly from the patient or from review of their medical records), different sampling strategies (eg, complete enumeration or random sampling), and different timings of the data collection (eg, continuously or at fixed points in time). These mechanisms have differing risks of bias.

In some CRXO trials, data are not directly sought from participants, but instead obtained from routinely collected sources (such as hospital medical records). In these trials, participants are identified (ascertained) but might not be approached directly for recruitment into the trial. Alternatively, participants could be recruited into the trial and might need to provide data prospectively, for example, through questionnaires. Recruitment can occur continuously or at fixed points in time. Recruitment or identification might involve approaching or identifying all participants or a sample.

Reporting the mechanisms for including individual participants, covering who was involved in recruiting or identifying participants, and whether they were blinded to the cluster’s treatment allocation in a particular period, allows assessment of the likelihood of bias arising from selective recruitment or identification of participants.54 In trials where participants are recruited or identified before randomisation, there is no potential for recruitment or identification bias (because selection of participants cannot be influenced by the cluster’s treatment allocation, which is unknown before randomisation). However, in CRXO trials, the possibility of using this approach is limited to only a closed cohort design (where the same participants are included in all periods), which is a less common design.8 More commonly, participants are identified or recruited after randomisation. In this circumstance, recruitment or identification of participants by a person unaware of the allocation of the cluster can help mitigate recruitment or identification bias. Trials with complete enumeration might have less scope for selective recruitment or identification of participants, even when the recruiter is aware of the cluster’s assigned treatment; nevertheless, concealing the treatment assignment from the recruiter is always advisable.58

Example of item 10b

Example 1 (continuous recruitment; recruiters unblinded to cluster’s treatment allocation)—“All adult patients were eligible, except those who stayed in the hospital longer than 21 days, who did not return home, who were in a moribund state, or who were not able to understand the topic of the study or complete a questionnaire.”39

“Participants are recruited by unblinded hospital pharmacists.” (Figure 1 of Pourrat et al.39)

Example 2 (random sampling)—“Because of the variability in cluster size, the female participants were only randomly selected for enrolment by probability proportional to size sampling. For the male participants’ enrolment, all the eligible participants were enrolled except for in three clusters whose size allowed random selection.”59

Explanation

Recruitment and consent procedures in cluster randomised trials are more complex than in individually randomised trials.60 In individually randomised trials, informed consent is usually obtained from individual participants, and consent generally encompasses being randomised, receiving study interventions, and providing data for outcome assessment. In cluster randomised trials, however, clusters are randomised—often before individual participants can even be identified for the trial, while study interventions and data collection procedures can be administered to different types of participants. Thus, the randomisation units, the units exposed to the intervention, and the units on which outcomes are assessed might be different. For example, hospitals could be randomised, the intervention could be given to providers, while outcomes are assessed on patients. Reporting of explicit details about the recruitment and consent procedures in CRXO trials is essential to avoid confusion. A simple statement that “participants provided informed consent” is not adequate; instead, referring to specific participants (eg, providers, patients) and indicating what consent pertains to (eg, exposure to the intervention or data collection) aids clarity.61 While so-called gatekeepers or cluster guardians can have an important role in facilitating the recruitment of clusters into the trial,62 gatekeeper permission for trial participation should not be confused with consent from research participants.14

Clarity around the timing of any consent relative to the timing of randomisation and intervention delivery is also particularly important in cluster randomised trials, because consent after randomisation increases the risks of bias.63 In a minority of CRXO trials (those with closed cohort designs), it might be feasible to identify and recruit individual participants before cluster randomisation, but most CRXO trials have cross sectional designs in which participants are recruited prospectively.8 In CRXO trials and in those CRXO trials with multiple periods, investigators should describe consent procedures in adequate detail so that it is clear whether consent procedures differed over time and between treatment conditions in different periods because these events can lead to bias.64 Steps taken to mitigate the risks of bias should also be clearly described.5458 Making recruitment materials and consent forms available in supplementary materials can aid clarity around what information was conveyed to participants about the trial, while providing timeline cluster diagrams can aid clarity around the timing of consent.55

In some CRXO trials in which outcomes are available in routinely collected databases and study interventions pose only minimal risk to research participants, a research ethics committee could give permission for a CRXO trial to proceed without the explicit informed consent of research participants. When an ethics committee has approved the study with a waiver of informed consent, this approval should be clearly stated together with the justification for the waiver according to the regulatory requirements of the country or countries where the CRXO trial took place.

Example of item 10c

Example 1 (consent procedures differ between treatment conditions for patients; timing of consent)—“6a. Participant recruitment in the medication reconciliation group . . . [Participants] receive complete information and provide oral consent for intervention and for data collection . . . 6b. Participant recruitment in the usual care group . . . [Participants] receive partial information because they are not aware of the existence of the medication reconciliation group and provide oral consent for data collection.” (Figure 1 of Pourrat et al.39)

Timeline cluster diagram—See supplementary materials 3, figure S1.39

Example 2 (waiver of consent to exposure to intervention)—“We conducted a cluster randomized multiple crossover trial to evaluate a real-time best practice alert embedded into electronic health records . . . Institutional review board approval was obtained before patient enrolment . . . Patient and provider consent was waived because the alert supplemented the current standard of care, there was no requirement to respond, and the alerts were unlikely to be harmful and might have proven beneficial.”65

Explanation

Clear reporting of whether those involved in the delivery of the treatments, receipt of the treatments, outcome collection and adjudication, and data analysis, were blinded to the treatment allocation is important for assessing the likelihood of bias. In some CRXO trials, participants are unaware of their treatment allocation because they do not know they are in the trial (eg, when they are not directly recruited), or they might know they are in a study, but not a trial.54 The level of awareness that participants have about their participation in a trial can affect the likelihood of bias. Therefore, explicit reporting of participants’ knowledge of their participation in the trial is important.

Performance bias could arise when healthcare providers or individual level participants are aware of the treatment allocation, and this knowledge leads to deviations from the intended treatment (eg, use of co-interventions, increased or decreased compliance with the treatment).41 Similarly, detection bias might arise when the outcome assessor or adjudicator is aware of the treatment allocation, which leads to differential assessment of the outcome. The outcome assessors can be cluster participants (eg, patients or healthcare providers that self-report outcomes), healthcare providers responsible for outcome assessment (eg, a clinical examination), trial personnel, or people who are independent of delivery of the treatment. Many trials will have different types of outcome assessors; therefore, clearly reporting which outcomes were assessed by whom, and whether they were blind to the treatment allocation is necessary.

Parallel and stepped wedge cluster randomised trials commonly evaluate treatments delivered to cluster level participants (eg, educational interventions delivered to healthcare providers to improve practice),6667 and in this circumstance, blinding of these participants would generally be difficult.54 However, in CRXO trials, treatments delivered at the individual participant level are more likely (because these treatments can be withdrawn—see item 5), and could provide more opportunity to use blinding of cluster level and individual level participants.8 The consequence of a lack of blinding of those people delivering the treatments (eg, clinicians) on performance bias might be of particular concern in CRXO trials. If changes in outcomes for individual level participants are observed in the first period by those delivering the treatment, this could lead to the continued use of the treatment condition in subsequent periods (a type of contamination).

Example of item 11a

Example 1—“Participants and ward staff were not blinded to treatment allocation.”34

“All primary and secondary outcome data collection and assessments were blinded to treatment allocation . . . Statisticians were blinded to treatment allocation until database closure.” (Supplement 3 of Kempen et al.34)

Example 2—“Blinding of clinicians and research staff will remain throughout the entirety of the trial . . . Trial participants, care providers, outcome assessors, and data analysts will be blinded to the assignment of intervention. The only research personnel who will be unblinded are the statistician performing the randomization, the PM [project manager], and the research pharmacists in charge of supplying locking kits used during the study . . . Data analysis will occur outside of the unit with the type of intervention received by each patient concealed until the completion of analysis.”68

Explanation

Primary reasons for reporting the statistical methods are to allow for reproduction of the results and for readers to assess whether the methods are appropriate for the design, and the research question. The guiding principle should be to provide enough detail such that a knowledgeable individual with access to the original data could verify the reported results.69 References to the chosen statistical methodology should be provided, along with details of the statistical software packages (and versions) used to implement the methods, and ideally the statistical code (items 24 and 27). The link between which statistical method(s) apply to which outcomes, should be made clear. Prespecified analyses should be distinguished from unplanned analyses. This distinction is important because prespecification limits bias arising from selective reporting of favourable results from fitting multiple analyses (commonly known as p hacking), and so prespecified analyses are considered more trustworthy.7071

Reporting the precise description of the treatment effect to be estimated for each outcome (ie, the estimand) allows readers to understand the specific questions asked,23 which should align with the study objectives (item 2b). The attributes for completely specifying an estimand are outlined in box 2. In addition to reporting the standard attributes for individually randomised trials (ie, attributes 1-5), investigators of cluster trials should report whether a participant average treatment effect or cluster average treatment effect was targeted (attribute 6).74 Attribute 6 will depend on the target of interest; that is, whether interest lies in how effective the intervention is on average for a population of participants (participant average treatment effect) or for a population of clusters (cluster-average treatment effect). Researchers should also specify whether interest lies in the marginal or cluster specific treatment effect.75

For cluster randomised trials, analyses can be undertaken at the individual or cluster level, with the second scenario done by aggregating observations within each cluster (or cluster period) into one summary measure. The approach taken in the trial’s analysis should be reported. Note that the level of analysis is distinct from the level of targeted inference.74 Recent literature concerning parallel cluster trials has indicated that participant average treatment effects and cluster average treatment effects might differ if there is informative cluster size, namely when cluster size is predictive of participant outcomes or of the magnitude of treatment effect (ie, treatment effect heterogeneity). In such cases, the different levels of analysis might be targeting different estimands.74 Although further work is needed to assess the impact of informative cluster size in CRXO trials, it should be reported whether informative cluster size has been considered or dealt with in the analysis.

It is well recognised that the statistical analysis of cluster trials must account for the correlation of participant observations within clusters (intracluster correlation). In CRXO trials, the strength of this correlation might depend on the separation in time between the observations,1776 along with (where applicable) correlation in repeated measurements from the same participants. Analyses that account for these correlations must be used: analyses that do not allow for these aspects typically yield incorrect estimates of standard errors of treatment effects (and therefore confidence intervals with coverage that is less or greater than the nominal level (eg, 95%)),10 and in some circumstances, might bias treatment effects (eg, certain generalised linear mixed models where the random effects are mis-specified).77 Therefore, reporting the estimation methods and assumptions made about the intracluster correlation of observations at different periods or times (eg, constant correlation, continuous-time correlation decay, exponential decay) is recommended.13

In CRXO trials where commencement of the trial is staggered across clusters or where period lengths differ across clusters, adjustment for secular or seasonal trends in the analysis should be considered, and description of the modelling approach should be provided (eg, including calendar time periods as a fixed effect).

Any tests for carryover effects, or adjustment for carryover, should be reported. The two stage analysis procedure, which involves first testing for carryover effects and determining the subsequent analysis approach based on the result of the test, has been shown to adversely affect estimation of the treatment effect and its statistical properties.78 For these reasons, testing for carryover is not recommended, but if undertaken, or an analysis of only the first period data are presented as a result of the two stage strategy, the analysis strategy needs to be reported. Similarly, when an attempt is made to adjust for carryover (eg, in designs with more than two periods), the model used and its assumptions should be described explicitly.79

Given that CRXO trials typically include a small number of clusters,8 small sample standard error or degrees of freedom adjustments might need be applied to maintain the nominal type I error rate (eg, 5%) and confidence interval coverage.80 Any adjustments applied should be reported. Similarly, researchers should report any adjustments for prognostic factors at the participant level to mitigate any imbalance in participant characteristics between periods within a cluster and note if they were prespecified or not.

Any methods for handling of missing data at the individual and cluster level should be described.8182 For trials with deviations from the allocated sequences or in the planned crossover timings, details should be provided about how this was dealt with in the analysis. The handling of these two components—missing data and treatment of deviations from allocated sequences—defines whether an analysis follows the intention-to-treat principle.383 However, challenges in applying the intention-to-treat principle in the analysis of parallel cluster randomised trials have been recognised.63 These issues are further exacerbated in multiple period cluster designs where different definitions of adhering to the allocated sequence might be adopted (eg, according to the planned dates of crossover, or to the order of the allocation sequence).12 For these reasons, explicitly reporting how deviations are handled and methods for missing data is preferable to using terms such as “intention to treat.”

For binary outcomes, reporting both absolute and relative effect sizes is recommended (item 17b). When a model based approach has been adopted, the method used to calculate the risk difference should be reported. Several methods exist, such as regression based approaches that directly yield estimates of risk differences (eg, fitting a binomial model with an identity link), or indirect approaches, such as marginal standardisation.84

Example of item 12a

Example 1 (individual level analysis; allowance for clustering, period and secular trends; method for calculating absolute effect)—“The analysis of the primary outcome used individual patient-level data and generalized estimating equations with a logarithmic link function, an exchangeable working correlation matrix, and robust standard errors using the ICU [intensive care unit] as the clustering unit. Because randomization was performed in batches of ICUs, covariate adjustment for randomization batch, the order of administration of the treatments, and batch × order interaction was performed to allow for separate order and secular time effects occurring in each of the randomization batches . . . Treatment comparisons are presented as risk ratios (RRs) and 95% CIs [confidence intervals] from the generalized estimating equation analysis, and as absolute risk differences and 95% CIs obtained by marginalizing (ie, standardizing) of the RR model [reference provided]. The approach to the analyses of the secondary end points appears in the [reference to supplement] . . . All statistical analyses were completed using Stata version 15 (StataCorp).”9

Example 2 (handling of missing data)—“The primary analysis will use all available data with no imputation. In addition, if the mRS [modified Rankin scale] at 90 days (the primary endpoint) or NIHSS [National Institute of Health Stroke Scale] at 7 days are missing for more than 10% of patients, missing data will be imputed using a fully conditional specification (FCS). The imputation model will include mRS at 90 days, the NIHSS and mRS scores at 7 days (or hospital discharge, if sooner), a variable indicating the cluster, a variable indicating the period, a variable indicating the intervention, and all baseline variables that are listed in Section 2.6 [sic] . . . The mRS at 90 days and NIHSS at 7 days will be imputed using an ordinal logistic model. Other variables will be imputed using either linear regression (for continuous/ordinal variables) or a discriminant function method (for nominal variables). Ten sets of imputed data will be created and analysed using the model described in Section . . . Estimates of the treatment effect (β in Model 1) and its standard-errors will be combined to obtain a pooled common OR [odds ratio] and 95% CI.” (Statistical analysis plan of reference 19.26)

Example 3 (cluster level analysis; allowance for period and prognostic factor)—“As primary analysis, the difference in risk of PJI [periprosthetic joint infection] after the intervention treatment compared with the control treatment will be estimated using linear regression for aggregated cluster-period data, with the proportion of events within a cluster-period as the dependent variable and treatment, cluster, period, and proportion of females within the cluster-period as independent variables. An estimated difference in risk with CIs [confidence intervals] and a 2-sided p-value will be presented.”85

Explanation

CRXO trials, like other designs, will commonly investigate subgroup differences and might perform adjusted analyses.3 Sensitivity analyses might also be undertaken to examine whether the results are robust to the assumptions of the analysis. These analyses deal with the same question (ie, estimand) but make different (plausible) assumptions that could lead to different results,86 for example, sensitivity of the treatment effect and its standard error to the choice of intracluster correlation structure. Details of such sensitivity analyses should be reported.

Example of item 12b

“A post-hoc series of analyses were undertaken to gauge the sensitivity of the point estimate and confidence interval from the primary analysis of in-hospital mortality to 90 days to alternative analysis models and small-sample corrections to the robust/sandwich variance estimator, and to the use of degrees of freedom from the t-distribution.” (Supplement 2 of PEPTIC trial.9)