Recommendations on recording harms in randomised controlled trials of behaviour change interventions

1.1: Recognise harm is possible from behaviour change interventions

There could be a misconception that harms from behaviour change interventions are not possible; a review of trial protocols identified that 10% did not record harms because “the intervention was behavioural and not expected to cause harm.”1 In the context of the Good Clinical Practice definitions of serious harm (eg, events resulting in death, life threatening episode, or admission to hospital),7 behaviour change interventions might be low risk. However, other important harms might need to be considered to fully examine the risks and benefits of the intervention. Evidence from our scoping review suggests that these types of interventions can cause harm.11 Most participants in our qualitative study were not aware of the literature identified in the scoping review. Therefore, discussion of examples of harm might be a first step in ensuring that the trial team understands that harm is possible from behaviour change interventions. Table 1 presents three examples of harm within different behaviour change interventions; further examples are available in our scoping review.11

1.2: Identify anticipated harms from an intervention: theorise, search the literature, and involve stakeholders

The Health Research Authority expect trial protocols to list harms expected from an intervention and within a population.8 However, a systematic review identified that only 15% of trial protocols did so in behaviour change trials.1 For drug trials, expected harms are contained in the reference safety information, such as the summary of product characteristics or investigator’s brochure. No such documents exist in non-drug trials, and therefore trialists might be uncertain in how to identify expected (which we refer to as anticipated) harms in behaviour change interventions. Trial teams might also need to consider if research procedures (eg, questionnaires) might cause harm.

Harms anticipated from an intervention can be best identified by following a three step process described by Bonell et al,2 known as the dark logic model. We provide a worked example in box 1. The three steps are:

1. Theorise: Expertise might exist within the trial team to apply social and psychological theories to consider how harms might occur.2 The systematic scoping review11 identified categories and mechanisms of harms (box A in supplementary material). We recommend that trial teams consider whether these categories and mechanisms apply to their intervention. For example, if an intervention is group based, a mechanism for harm could lead to unhelpful knowledge exchange among trial participants, which might promote undesirable behaviours.

2. Search the literature: The literature is unlikely to cover the exact intervention being evaluated in the trial. Use a broad literature search by considering similar interventions. Owing to the paucity of evidence of harms within behaviour change interventions in randomised controlled trials, trial teams might need to consider other study designs. Search terms will need to reflect how authors might describe harm (eg, “unintended”/“unanticipated” or “unwanted”/“unplanned”; “event”/“effect”(s), “harm”/“consequence”(s), or “impact”/“repercussion”(s)).

3. Include stakeholder input: Stakeholder perspectives are important for identifying potential harms and determining the importance of those harms. These stakeholders might include patient and public representatives or those delivering the intervention (such as schoolteachers). Sharing examples of harms from previous behaviour change interventions might be helpful, even if not related to the intervention being evaluated, to demonstrate the types of harms possible and provide context.

1.3: Consider if anticipated harms are captured by the Good Clinical Practice definition of harm

Our qualitative study identified the current approach to recording harms in behaviour change intervention trials is driven by regulatory requirements and Clinical Trials Unit Standard operating procedures. This process involves defining harms as adverse events. An adverse event is any untoward medical occurrence in a patient or clinical investigation participant given a pharmaceutical product and that does not necessarily have to have a causal association with this treatment. Each adverse event is assessed for seriousness, expectedness, and relatedness to the intervention. A serious adverse event is an adverse event that at any dose results in death or is life threatening (ie, requires hospital admission or a prolongation of existing hospital stays, results in persistent or clinically significant disability or incapacity, or is a congenital abnormality or birth defect). This assessment identifies serious adverse reactions and suspected unexpected serious adverse reactions, with the second group of events being reportable to regulatory authorities according to strict timelines.

The main problems with the current approach are that it could miss relevant and important harms that are not medical events or consequences, the medical terminology might not be appropriate for all harms that arise from behaviour change interventions,12 and it might be highly inefficient owing to excessive time spent recording large numbers of irrelevant events.91012

Once trial teams have identified anticipated harms from a behaviour change intervention (see section 1.2), we recommend that trial teams consider whether these harms are captured by the Good Clinical Practice definitions.7 Harm might not always be a medical event or consequence, as demonstrated by the examples in table 1. Furthermore, the
Consolidated Standards of Reporting Trials (CONSORT) harms extension defines harm as “the totality of possible adverse consequences of an intervention or therapy; they are the direct opposite of benefits, against which they must be compared,”19 and recommends against using terms such as “adverse events” and “side effects.”

Researchers might be uncertain as to what constitutes an adverse consequence, according to the CONSORT harms extension definition.19 We identified three key aspects that might help define harm arising from a behaviour change intervention.1112 Firstly, the harm must be plausibly caused by the intervention. Secondly, perspectives beyond just the participant might be important to consider, such as family members or trial personnel. Thirdly, owing to potential subjectivity on what may be viewed as harmful, an event must be “of concern” to the participant or other relevant person for it to be considered harmful. Subjectivity is inherent in harm perception, because individuals might view the same event differently, and vary in their response (ie, some individuals might think of an event as a minor discomfort, whereas others might be more concerned). Therefore, defining what is “of concern” is difficult; participants in our qualitative study found this process to be complex. Involving patient and public representatives could provide insight into perceived importance of harms.

We propose the following definition of harm within a behaviour change intervention trial: “An event or an unintended consequence plausibly caused by a trial intervention or research procedure, and which is of concern to a study participant or other relevant person.” Note that consent will be required if there is collection of harms from non-trial participants, for example, significant others or family members.

1.4: Identify the common harmful events within trial populations

Harmful events inherent in a trial population (eg, hospital admissions for medical events occurring in an elderly population) often pose difficulties for trial teams in terms of whether such events should be recorded as a harm within a trial. Initially, we recommend that the trial team identifies the common harmful events within their trial population. Deciding on whether these events are to be recorded as harms within a randomised controlled trial can be considered as part of section 1.5 (proportionate recording).

1.5: Proportionate recording of harms: consider prioritising the most plausible or important harms

At this point, trial teams will have identified a list of anticipated harms that might or might not be captured using the Good Clinical Practice definition of harms.7 Trial staff can spend a disproportionate amount of time recording irrelevant events.91012 Instead, trial teams could prioritise the most plausible, serious, and important harms to conserve trial resources.

Determining what events to prioritise or exclude in recording of harms is likely to be on a trial-by-trial basis, with multidisciplinary team input including patient and public involvement representatives. A reasonable explanation should be given for how the harm could be caused by the intervention (ie, plausibility). Decisions on what information to record might also consider the exclusion of events that occur frequently in both trial arms with limited evidence of causation from the intervention. Furthermore, a balance should be struck between risk of omission of harms and the burden on the trial team. Where a plausible link cannot be ruled out, and if such events are expected and serious in nature, a more cautious and conservative approach might be required because the risk of omission is high. For example, hospital admission for children and adolescents might be recorded in a psychological therapy trial. The low frequency of such events will not be burdensome to the trial team.

Decisions to exclude events from harm recording should have approval from the oversight committee and sponsor. Events not recorded as harms might still need to be reported through other trial protocols, such as safeguarding. Box 2 provides a worked example.

1.6: Ensure transparency in recording harms: list anticipated harms in protocol as a minimum standard

The scoping review identified a need for better transparency in recording of harms. At a minimum, researchers should list plausible expected harms from the trial intervention or research procedures and harms expected in the population in the trial protocol. This is an expectation of the NHS Health Research Authority.8 We also recommend, as best practice, to document the methods used to identify harms plausibly caused by the intervention (eg, by the three step approach of the dark logic model2), and identify which harms are excluded from recording (with justification for doing so). This information could be included in the trial protocol or a standard operating procedure and can demonstrate that trialists have not cherry picked harms at the time of trial reporting.20 The sharing of harms data is also important. Rare and unpredictable harms are unlikely to be identified in one randomised controlled trial, but synthesising harms data across studies could enable identification of such events. However, the lack of consistency in harms reporting, even in the same health condition and same intervention, makes synthesising difficult.21

2.1: Use a range of data collection methods

The systematic scoping review identified that harms should be collected systematically by a range of data collection methods.1121 These methods include the trial outcome measures intended to measure efficacy, direct questions, open ended questions, qualitative research, and bespoke instruments to record harms,222324252627282930 where they exist.

Trial outcome measures recording efficacy of the intervention offer an opportunity to capture harm. For example, where an intervention might plausibly worsen depression, a trial outcome measure such as a depression scale would detect this change. This approach relies on a data monitoring committee to view the accumulating data, although behaviour change intervention trials are frequently termed as low risk, and might not have such a committee. The literature has examples of interventions worsening the health behaviour intended for improvement.23 Therefore, an independent oversight committee with access to the blinded data should be considered (sometimes a trial steering committee can perform this role).

Anticipated harms should be recorded systematically,21 by formulating a direct question or anticipated harms specified in the protocol. Naming specific events that might prime or influence a participant and introduce bias might raise concerns. However, behaviour change intervention trials routinely use patient reported outcome measures, a limitation that has the potential to introduce self-report bias.

Unanticipated harms should be considered because it is impossible to consider all potential harms that are plausible from an intervention at trial outset.1131 No approaches were found in the literature or qualitative study on how to do this. We suggest prompting participants with an open ended question (see example in box 3) or using qualitative research,1112 especially during the early stages of intervention evaluation (eg, feasibility studies or external pilots). Researchers should avoid any inadvertent unblinding of staff involved in reviewing open ended questions.

2.2: Monitor and adapt the harms recorded during the trial if necessary

Flexibility is important,11 which might entail regular review of harms recorded during the trial to flag modifications required. For example, over-recording of events not plausibly caused by an intervention could occur. Conversely, trial teams might become aware that important harms are not being collected. Trial oversight committees can review and agree proposed adaptations to the recording of harms.

2.3: Train trial staff in the complexities and importance of recording harms

The trial team will need to understand the complexities and importance of recording harms.11 Examples of harms that fall outside the Good Clinical Practice definition might be useful (table 1; further examples available in our scoping review11). Where there is potential for data collectors to influence the reporting of harms data, for example, if sensitive harms data or potential for power imbalances exist (eg, in a trial of children and young people), the importance of an empathetic approach could be highlighted during training.

2.4: Consider the potential for reporting bias between trial arms

Considering the time points for data collection could reduce reporting bias.11 Harms data collection could be imbalanced between the trial arms, for example, where the intervention arm has more study visits. For remotely delivered interventions, opportunities to collect harms might also be limited.

2.5: Attempt attribution assessment wherever possible

There are limitations to assessing whether an individual event is related to an intervention or trial procedure. Researchers (who we interviewed) described difficulties in obtaining complete data on events to make attribution assessments, particularly where repeated requests for further information to assist with attribution assessment was not appropriate (eg, where an event was sensitive or distressing).12 Input from several trial team members might be required, which can be resource intensive. These difficulties do not have easy solutions, but we recommend where possible that attribution of events is attempted. Trial oversight committees (data monitoring or trial steering committees) can help assist with attribution assessment where an assessment cannot be made by the trial team. The difficulties in assessing whether an individual event or harm is related to the intervention underlines the importance of recording harms in both the intervention and control arms. In doing so, this will allow comparison on the rate of harms, as would be expected in any trial irrespective of intervention type.