Intended for healthcare professionals

  1. Research
  2. Coaching inexperienced...
  3. Coaching inexperienced clinicians before a high stakes medical procedure: randomized clinical trial
CCBYNC Open access
Research Christmas 2024: Could We Start Again, Please?

Coaching inexperienced clinicians before a high stakes medical procedure: randomized clinical trial

BMJ 2024; 387 doi: https://doi.org/10.1136/bmj-2024-080924 (Published 16 December 2024) Cite this as: BMJ 2024;387:e080924

Linked Editorial

Just-in-time training could be just what the doctor ordered
  1. Stephen G Flynn, instructor and associate in perioperative anesthesia1 2,
  2. Raymond S Park, assistant professor and senior associate in perioperative anesthesia1 2,
  3. Anupam B Jena, Joseph P Newhouse professor of health care policy and medicine3 4 5,
  4. Steven J Staffa, biostatistician1 2,
  5. Samuel Y Kim, clinical research specialist2,
  6. Julia D Clarke, clinical research assistant2,
  7. Ivy V Pham, clinical research specialist2,
  8. Karina E Lukovits, clinical research specialist2,
  9. Sheng Xiang Huang, clinical research assistant2,
  10. Georgios D Sideridis, assistant professor and senior biostatistician6 7,
  11. Rachel S Bernier, research manager2,
  12. John E Fiadjoe, associate professor and executive vice chairman of anesthesia1 2,
  13. Peter H Weinstock, professor and anesthesia endowed chair in pediatric simulation and senior associate in critical care medicine1 2,
  14. James M Peyton, assistant professor and senior associate in perioperative anesthesia1 2,
  15. Mary Lyn Stein, assistant professor and associate in cardiac anesthesia1 2,
  16. Pete G Kovatsis, associate professor and senior associate in perioperative anesthesia1 2
  1. 1Department of Anesthesia, Harvard Medical School, Boston, MA 02115, USA
  2. 2Department of Anesthesiology, Critical Care, and Pain Medicine, Boston Children’s Hospital, Boston, MA, USA
  3. 3Department of Health Care Policy, Harvard Medical School, Boston, MA, USA
  4. 4Department of Medicine, Massachusetts General Hospital, Boston, MA, USA
  5. 5National Bureau of Economic Research, Cambridge, MA, USA
  6. 6Department of Pediatrics, Harvard Medical School, Boston, MA, USA
  7. 7Institutional Centers for Clinical and Translational Research, Boston Children’s Hospital, Boston, MA, USA
  1. Correspondence to: S G Flynn stephen.flynn{at}childrens.harvard.edu
  • Accepted 27 October 2024

Abstract

Objective To assess whether training provided to an inexperienced clinician just before performing a high stakes procedure can improve procedural care quality, measuring the first attempt success rate of trainees performing infant orotracheal intubation.

Design Randomized clinical trial.

Setting Single center, quaternary children’s hospital in Boston, MA, USA.

Participants A non-crossover, prospective, parallel group, non-blinded, trial design was used. Volunteer trainees comprised pediatric anesthesia fellows, residents, and student registered nurse anesthetists from 10 regional training programs during their pediatric anesthesiology rotation. Trainees were block randomized by training roles. Inclusion criteria were trainees intubating infants aged ≤12 months with an American Society of Anesthesiology physical status classification of I-III. Exclusion criteria were trainees intubating infants with cyanotic congenital heart disease, known or suspected difficult or critical airways, pre-existing abnormal baseline oxygen saturation <96% on room air, endotracheal or tracheostomy tubes in situ, emergency cases, or covid-19 infection.

Interventions Trainee treatment group received preoperative just-in-time expert intubation coaching on a manikin within one hour of infant intubation; control group carried out standard practice (receiving unstructured intraoperative instruction by attending pediatric anesthesiologists).

Main outcome measures Primary outcome was the first attempt success rate of intraoperative infant intubation. Modified intention-to-treat analysis used generalized estimating equations to account for multiple intubations per trainee participant. Secondary outcomes were complication rates, cognitive load of intubation, and competency metrics.

Results 250 trainees were assessed for eligibility; 78 were excluded, 172 were randomized, and 153 were subsequently analyzed. Between 1 August 2020 and 30 April 2022, 153 trainees (83 control, 70 treatment) did 515 intubations (283 control, 232 treatment). In modified intention-to-treat analysis, first attempt success was 91.4% (212/232) in the trainee treatment group and 81.6% (231/283) in the control group (odds ratio 2.42 (95% confidence interval 1.45 to 4.04), P=0.001). Secondary outcomes favored the intervention, showing significance for decreased cognitive load and improved competency. Complications were lower for the intervention than for the control group but the difference was not significant.

Conclusions Just-in-time training among inexperienced clinicians led to increased first attempt success of infant intubation. Integration of a just-in-time approach into airway management could improve patient safety, and these findings could help to improve high stakes procedures more broadly. Randomized evaluation in other settings is warranted.

Trial registration ClinicalTrials.gov NCT04472195.

Introduction

The successful performance of physical tasks is critical in many occupations, including sports, music, aviation, and medicine. Like doctors,123 athletes and musicians at all levels practice many hours with structured coaching to attain expertise.4 Unlike in medicine, these professions also universally rehearse right before a performance, or just in time, with coaches who review mechanics, approach, and mental engagement to optimize outcomes.567 For example, although a professional football goalkeeper practices many hours with a coach in training, right before the match, the coach takes the field with the keeper, implementing a regimented shooting drill integrated with situational preparation to maximize performance for the day’s opponent. The drill is structured around areas of weakness for that goalkeeper.

Therefore, it is surprising that in medicine, an industry with one of the highest stakes where performing a procedure can have life-altering consequences, just-in-time training89 is rare to non-existent. This deficit is potentially most important for inexperienced clinicians: those who are not only asked to perform high risk tasks at the limit of their manual and cognitive abilities, but also lack the cumulative experience and task familiarity on which to rely. Among these clinicians, receiving training weeks before a procedure is ultimately performed might be less optimal than receiving training days or even minutes before.101112

An example of how just-in-time training might improve outcomes of high stakes procedures is intubating infants and newborn babies. One million infants have surgery in the US annually, of whom many are intubated by trainees.13 Most intubations are via intraoperative guided instruction by senior anesthesiologists who allow the trainee to intubate the infant with no pre-training, sometimes leading to multiple intubation attempts, which are associated with severe complications, including hypoxia, bradycardia, and cardiac arrest.1415161718 Infants are particularly vulnerable during intubation because of their rapid oxygen desaturation,19 which creates time pressure and increases clinician cognitive load.20 Intubating the infant on the first attempt is a crucial patient safety metric,2122 and just-in-time training could, in theory, improve the performance of an inexperienced clinician.

Therefore, we conducted a randomized clinical trial to assess whether coaching inexperienced clinicians just before a procedure could improve the quality of procedural care. Specifically, we examined whether just-in-time training by an expert airway coach within one hour of clinical care would improve the first attempt success rate of inexperienced clinicians performing infant intubation. We also assessed the impact of just-in-time training on complications, trainee cognitive load during intubation, and procedural competency.

Methods

Study design and oversight

This single center, prospective, non-crossover, parallel group, non-blinded, randomized clinical trial was conducted at Boston Children’s Hospital, a large quaternary academic medical center in Boston, MA, USA. The hospital’s institutional review board (P00034169) approved the study. The trial was registered with ClinicalTrials.gov (NCT04472195) and was conducted according to the Declaration of Helsinki.

Participants

Participants were anesthesiology trainees from 10 regional training programs doing pediatric anesthesia rotations at Boston Children’s Hospital. Trainees comprised fellows (doctors who have completed residency and are pursuing one year of advanced training to become a subspecialty attending physician anesthesiologist, here pediatric anesthesia), residents (doctors who graduated medical school and are pursuing general specialty training, here anesthesia, a three year program), and student registered nurse anesthetists. A student registered nurse anesthetist is a registered nurse in the US who has graduated from nursing school and works for at least one year as an intensive care unit nurse; they undertake a three year program to become a certified registered nurse anesthetist. Trainees were all directly supervised by attending anesthesiologists (equivalent to consultant anesthetists in the UK and other regions) for consideration of newborn or infant cases. Informed consent was obtained before trainee participation.

Research assistants approached and obtained informed consent from these trainees at the hospital in person during the first day of their pediatric anesthesia rotation orientation or via a remote consent system (approved by the institutional review board) during the covid-19 pandemic. No trainees had access to infant manikins to practice before study involvement. Inclusion criteria were eligible trainees performing orotracheal intubations in infants aged ≤12 months with an American Society of Anesthesiology physical status classification of I-III. Exclusion criteria were anesthesia trainees assigned to infants with known cyanotic congenital heart disease, infants with known or suspected difficult or critical airways, infants with pre-existing abnormal baseline oxygen saturation <96% on room air, infants with endotracheal or tracheostomy tubes in situ, emergency cases that would start within one hour of the booking, and infants with covid-19 infection.

Randomization and masking

We stratified trainees by role (fellow, resident, student registered nurse anesthetist (SRNA)) and prospectively block randomized to the treatment or control group for tracheal intubation of children aged ≤12 months. The random allocation sequence and randomization schedules were created by the study statistician, who had no part in patient enrollment, using the PROC PLAN procedure in SAS (version 9.4, SAS Institute, Cary, NC). A randomized block size of four was implemented to ensure that every set of four randomized trainees would be equally randomized to the treatment or control groups. The study coordinators and research assistants followed the randomization schedule, enrolled participants, and kept a study screening and enrollment log.

Procedures

On enrollment, trainees answered an infant intubation self-assessment questionnaire. The research team identified eligible encounters and contacted the attending anesthesiologist and trainee before their case. The treatment group received a just-in-time coaching session before each clinical intubation encounter. The control group, according to our routine practice, had unstructured intraoperative (rather than preoperative) instruction in intubation by attending pediatric anesthesiologists. For the treatment group, intubation equipment (laryngoscope type) and technique (direct laryngoscopy v video laryngoscopy) were chosen by the intraoperative case attending anesthesiologist and communicated to the attending anesthesiologist conducting the coaching session. The intraoperative case attending anesthesiologist and coach were always different providers. The treatment group received a standardized coaching session (supplementary figure S1) on an infant manikin within one hour of patient intubation in the perioperative simulation suite (provided by one of five attending anesthesiologist airway coaches). The approach included critical intubation steps. Flexibility existed in these sessions to correct trainee specific issues. In addition, three coaching insights (supplementary figure S1) dealing with potential intubation challenges were taught during the session. Treatment group trainees completed two successful manikin intubations or 10 minutes of training before their patient encounter.

A research team member observed intraoperative intubation attempts for treatment and control groups. To ascertain the cognitive load of each intubation, the trainee immediately filled out the unweighted National Aeronautics and Space Administration task load index (NASA-TLX) after intubation.23 NASA-TLX is a short, highly reliable (Cronbach α coefficient >0.80), and valid, Likert survey of six questions that was developed by NASA to measure mental workload while performing a task—defined as the cost incurred by a human operator to achieve a performance level (supplementary figure S2). Each performer has a cognitive workload limit,20 and high NASA-TLX scores (ie, high cognitive load) have been correlated with increased task specific error.2024252627 The six specific standardized domains measured were trainee self-perception of mental, physical, and temporal demands and performance; effort; and frustration related to the intubation encounter. Ongoing research suggests that scores of 50 or higher could indicate a high cognitive load.282930

Trainees performed up to five patient intubations, and trainees in the treatment group received a coaching session before each intubation to allow for a competency acceleration analysis (ie, an evaluation of how quickly competency was achieved). Study procedures were completed after the five observed intubations or completion of the clinical rotation. Trainees in the control group were offered coaching sessions to allow an equal opportunity for learning at the end of the study period. The protocol included video capture of the training sessions for qualitative analysis. However, capturing video during case turnover proved unfeasible and was abandoned early in the study.

Outcomes

The primary outcome was the first attempt success rate of intraoperative tracheal intubation of infants. An attempt was defined as laryngoscope blade insertion into the mouth to blade removal from the mouth. Attempt time was from blade insertion until sustained expired carbon dioxide was detected. Secondary outcomes included complication rates; cognitive load of intubation (measured by NASA-TLX); and competency metrics, including time to intubation, airway view defined by modified Cormack-Lehane scoring3132 (a grading system based on the extent of laryngeal anatomy visible during intubation that predicts intubation ease or difficulty; supplementary figure S3), advancement maneuvers (number of times the trainee tried to place the breathing tube in the airway on a given intubation attempt), and technical difficulties. We also assessed if competency, defined by successful first attempts at intubation, occurred earlier on average over five successive intubations among trainees who received just-in-time training.

We analyzed complications, categorized as severe or non-severe according to definitions from the Pediatric Difficult Intubation Collaborative.21 Complications were limited to the airway encounter. There was no long term follow-up. Non-severe complications included mild hypoxemia (saturation <90% but >80%), laryngospasm, bronchospasm, minor airway trauma (dental or lip), and airway activation. Severe complications included moderate hypoxemia (saturation ≤80% but >50%), severe hypoxemia (lowest saturation ≤50%), esophageal intubation, cardiac arrest, and pharyngeal bleeding.

Statistical analysis

The study was powered for the primary outcome of first attempt intubation success. We assumed an 80% first attempt success rate to determine the sample size. This assumption was based on an internal data query of the previous five years of trainee intubations and published literature.1433 Using a χ2 test with a 5% two sided α, a sample size of 200 intubations per group (400 total) provided 80% power to detect a difference of 80% versus 90% (10% absolute difference) in first attempt success rates. We planned for 500 total intubations to account for potential attrition. No interim analyses were planned, nor were stopping guidelines included because we reasoned that additional coaching would not lead to increased harm. An intention-to-treat and per protocol analysis was planned.

Categorical data were presented using frequencies and percentages, and continuous data were presented using means and standard deviations or medians and interquartile ranges (IQR). Denominators were presented to indicate variables with missing data, and all non-missing data were included in the analysis. The balance between the randomized groups was evaluated for each variable using the standardized mean difference, where values less than 0.2 were considered to represent a good balance between arms.

Multivariable generalized estimating equations modeling was implemented with a logit link (to estimate odds ratios) or log link (to estimate risk ratios) and binomial family to account for multiple intubations per participant to compare the primary outcome of success rates between groups. We adjusted multivariable models for baseline variables and potential confounders with standardized mean differences greater than 0.2, comparing the groups. Fisher’s exact test was implemented for outcomes where odds ratios cannot be calculated. We analyzed continuous secondary outcomes using median regression with a random effect for trainees. Groups for the NASA-TLX were compared as a continuous score using generalized estimating equations modeling with an identity link and Gaussian family. We applied mixed effects, ordinal logistic regression for specific Likert scale domains and ordinal outcomes. Results from multivariable adjusted regression analyses were presented as adjusted odds ratios or coefficients with corresponding 95% confidence intervals (CI) and P values. Denominators were displayed to indicate variables with missing data. We stratified subgroup analysis by trainee type (fellow, resident, SRNA). A two tailed P<0.05 value was to be considered significant. We did a number-needed-to-treat analysis at the number of intubations level. Data were managed with REDCap and stored on a secure server. We did statistical analyses using Stata (version 16 0.0, StataCorp, College Station, TX).

Patient and public involvement

We generated our research question from our previous infant airway research studies, our education work with trainees, unanswered questions regarding best practices in training, and our simulation center experiences and related literature. As the trainee was the study participant for the clinical trial and the study commenced before patient and public involvement was common, the public was not involved in setting the research question.

Results

Between 1 August 2020 and 30 April 2022, 250 trainees were assessed for eligibility (fig 1), of whom 172 trainees were randomized (89 control, 83 treatment). Five trainees were withdrawn after randomization. Reasons for withdrawal included extensive prior experience with infant intubation, a need for remediation, and incidental enrollment of trainees participating in a cardiac fellowship that was an excluded subgroup. Fourteen trainees (10 in the treatment group and four in the control group) were excluded from the final analysis because they did not intubate a study infant. Therefore, 153 trainees (83 control, 70 treatment) received the intended study protocol and were analyzed via modified intention-to-treat and per protocol.

Fig 1
Fig 1

Consort diagram. After exclusions, randomization and analysis included 70 trainees in the treatment arm (45 residents, 15 fellows, and 10 student registered nurse anesthetists (SRNAs)) and 83 trainees in the control arm (53 residents, 16 fellows, and 14 SRNAs)

Overall, 515 intubations were performed (283 control, 232 treatment) and analyzed by the originally assigned groups. The initial protocol planned enrollment of 100 trainees to do five intubations each, because the study was powered for 500 intubations. However, we later discovered that we were not achieving five intubations for all trainees. We made an institutional review board amendment to enroll more trainees to achieve the targeted number of study intubations. Therefore, recruitment was stopped after 172 trainees—the minimum projected number to attain 500 intubations. The trial was completed after all enrolled trainees completed the study protocol, yielding 515 intubations.

Demographic and baseline characteristics were similar, except for trainees in the control group performing direct laryngoscopy on the first attempt more often than trainees in the treatment group (table 1). A subgroup analysis was performed to determine the impact of this difference (table 2).

Table 1

Demographics and baseline characteristics of randomized groups, by trainee and intubation level data

View this table:
Table 2

Analysis of first attempt success at infant intubations

View this table:

Seven protocol deviations occurred in the treatment group, which were instances when more than one hour had occurred between coaching and intubation of the patient (n=2), or an alternate approach was used for patient intubation rather than for the coaching session (n=5). Given the small number of protocol deviations, we reported only the modified intention-to-treat analysis. This intention-to-treat analysis was a modified analysis—that is, we analyzed all available data based on the randomized trainee groups; however, some trainees did not perform any study intubations and were therefore removed from the study analysis (as indicated in figure 1).

Primary outcome

Overall, first attempt success for tracheal intubation was higher in the treatment group than in the control group (91.4% (212/232) v 81.6% (231/283), odds ratio 2.42 (95% CI 1.45 to 4.04), P=0.001; table 2). The number needed to treat for the primary outcome was 10.2 (95% CI 6.4 to 25.2) at the number of intubations level. Residents had a first attempt success of 93% (132/142) in the treatment group versus 81.4% (140/172) in the control group (odds ratio 3.18 (95% CI 1.62 to 6.24), P=0.001). Fellows had a first attempt success of 90.5% (67/74) in the treatment group versus 85.9% (67/78) in the control group (1.57 (0.56 to 4.41), P=0.39). For student registered nurse anesthetists, first attempt success was 81.3% (13/16) in the treatment group versus 72.7% (24/33) in the control group (1.82 (0.53 to 6.24), P=0.34). An adjusted comparison of first attempt success by study arm is shown in figure 2. We saw significantly higher odds of success among all trainees for the treatment group than for the control group (2.42 (1.45 to 4.04), P=0.001). Furthermore, odds of success among residents for the treatment group was significantly higher than for the control group (3.18 (1.62 to 6.24), P=0.001).

Fig 2
Fig 2

Adjusted comparison of first attempt success of infant intubation by study arm. Data are adjusted odds ratio (95% confidence interval) for treatment group (reference=control group). SRNA=student registered nurse anesthetist

We performed a primary outcome subgroup analysis of intubations involving video and direct laryngoscopy (table 2). When video laryngoscopy was used, first attempt success was 91.9% (193/210) in the treatment group versus 81.9% (181/221) in the control group (odds ratio 2.58 (95% CI 1.48 to 4.5), P=0.001). When direct laryngoscopy was used, first attempt success was 86.4% (19/22) in the treatment group versus 80.7% (50/62) in the control group (1.61 (0.42 to 6.2), P=0.49).

Secondary outcomes

The overall complication rate was 2.75% (7/255) in the treatment group and 4.71% (16/340) in the control group (odds ratio 0.57 (95% CI 0.23 to 1.41), P=0.22; table 3, supplementary table S1). Mental workload scores, measured by the NASA cognitive task load index, were significantly lower for mental demand (coefficient −9.5 (95% CI −16 to −3), P=0.004), temporal demand (−9.1 (−16.1 to −2.1), P=0.01), effort (−10.1 (−16 to −4.4), P=0.001), and frustration (−7.1 (−12.6 to −1.7), P=0.01) in the treatment group. We saw no differences between groups for the physical demand (−3.2 (−8.2 to 1.8), P=0.21) and performance (−2.9 (−6.6 to 0.7), P=0.11) domains (supplementary tables S2 and S3; supplementary figures S4 and S5).

Table 3

Comparison of secondary outcomes relating to infant intubations

View this table:

A competency acceleration analysis—designed to measure whether just-in-time training expedited trainee skill acquisition—showed a significant difference between groups in first attempt success rates by intubation rounds two and three, favoring the intervention (table 2). For example, in round two, by which the treatment group had received their second just-in-time training, their first attempt success rate was 90.9% (50/55) versus 73.9% (51/69) in the control group (odds ratio 3.53 (95% CI 1.22 to 10.2), P=0.02). By round three, after three just-in-time training sessions, the treatment group’s first attempt success rate was 95.8% (46/48) versus 77.6% (45/58) for the control group (6.64 (1.42 to 31.1), P=0.02). Competency acceleration dropped off in intubation rounds 4 and 5, with higher uncertainty of the estimates. This drop-off might be due to relatively smaller sample sizes in each study group for trainees with a fourth and fifth intubation round, leading to more variability and uncertainty of the effect estimates. Gradual progress and improvement were seen in the control group, with similar success rates by round 5.

We saw significant quantitative differences in technical skill metrics between groups. The treatment group had more modified Cormack-Lehane grade 1 views (the best possible airway view) for video laryngoscopy than the control group, half the number of endotracheal tube advancement maneuvers, fewer technical difficulties during laryngoscopy, and faster intubation times (table 3, supplementary table S1). The direct laryngoscopy findings were not significant. The direct laryngoscopy modified Cormack-Lehane grade 1 view favored the control group (table 3, supplementary table S1). Grade 2A views, which still have a high probability of successful intubation, were similar between direct laryngoscopy groups. No adverse events related to this study were reported to the study coordinator or institutional review board.

Discussion

This randomized clinical trial demonstrated that just-in-time training was associated with significantly improved first attempt success of infant orotracheal intubation by pediatric anesthesia trainees. The improvement in first attempt success by 10 percentage points is clinically meaningful, considering the harms associated with multiple tracheal intubation attempts and the many trainee intubations performed yearly. Just-in-time training was associated with significant process improvements in quality of care, including decreased time to intubation, improved views of the larynx while intubating (which leads to easier breathing tube insertion), fewer advancement maneuvers in placing the breathing tube, and fewer technical difficulties. Finally, just-in-time training was associated with a significantly lower cognitive task load while intubating. Lower cognitive task loads are associated with fewer task specific errors, which is crucial in the potential morbidity and mortality associated with infant intubation. Our findings indicate that just-in-time training could improve clinical outcomes in high stakes medical procedures, particularly among inexperienced clinicians.

The observed improvement in trainee skills and cognitive workload is important and timely. In recent studies, infants who needed more than two attempts to intubate had 40% incidence of hypoxemia and 8% incidence of bradycardia.15 Newborn babies and infants comprise a quarter of cardiac arrests due to respiratory causes (including failed intubation) in the recent UK National Audit (NAP7) on perioperative cardiac arrest.16 In our study, complication rates were lower in the treatment group than in the control group, although this difference was not significant. However, our study was powered for first attempt success of intubation and not complications. Complications in the treatment group were half of those in the control group, which is clinically meaningful.

Our exploratory primary outcome findings suggest that just-in-time training before procedures improves infant intubation outcomes even in a setting where, moments later, intraoperative attending anesthesiologists can provide real time bedside teaching using video laryngoscopy, as was allowed in the control group. Video laryngoscopy is rapidly becoming the standard of care34 in infant intubation as it improves first attempt intubation success.2135 The approach allows attending anesthesiologists to guide trainees while intubating as they can, in real time, share a video screen of the airway with the trainee.3536 Despite this enhanced ability to provide trainees with real time feedback during intubation, trainees in the treatment group still had higher first attempt success rates and better airway views when using intraoperative video laryngoscopy than trainees in the control group, who had no pre-coaching and only intraoperative instruction. Expert guided coaching37 just before clinical care could, therefore, expedite competency and prime trainees for the clinical encounter.

Comparison with other studies

While our study focused on pediatric intubations, our findings indicate that just-in-time training might improve the quality of procedural care more generally. Although to rehearse, warm up, or practice is standard before performing in several high stakes occupations, just-in-time training is rare to non-existent in medicine. Previously, just-in-time simulation for tracheal intubation in the pediatric intensive care unit was compared to historical controls, and no difference in first attempt success rate was found. However, that study was not randomized, and in it, training could occur up to 24 hours before the clinical encounter, compared with training that occurred within an hour of intubation in our study.38 A just-in-time lumbar puncture cohort study for pediatric interns also did not show improved clinical success rates,39 but the study was non-randomized, the intervention supervisors included all attending physicians and senior house staff rather than a specific coaching team, and participants surveyed reported that teaching and supervisor engagement was highly variable and that the intervention was sometimes skipped if perceived as a barrier to workflow.40 In contrast, a prospective, randomized trial on intubation in the neonatal intensive care unit showed a significant clinical effect of just-in-time simulation versus video intubation education among junior pediatric residents.41 Although that study was smaller and did not have a dedicated coaching team, their findings are consistent with ours and demonstrate the potential generalizability of just-in-time training and the importance of engaging motor skills as part of the warm-up for inexperienced clinicians.

Policy and research implications

Although our study focused on just-in-time training for inexperienced clinicians, our findings raise whether experienced clinicians might benefit. Indeed, just-in-time training and physical warm-up in other professions are ubiquitous and not only restricted to those without experience. Whether this same principle applies to experienced clinicians is an open question. While it is well established within medicine that greater clinical volume is associated with better clinical outcomes (ie, the volume-outcome relationship42), the timing of when that volume accrues also likely matters. More recent experience might be associated with improvement in procedural outcomes, managing human capital depreciation, or skill decay. A study of high volume cardiac surgeons found that even small temporal breaks in surgical care (ie, days away from the operating room) affected surgeons’ performance in coronary artery bypass graft surgery.43 This finding suggests that just-in-time practice, with or without coaching, could improve procedural outcomes, an area of future study.

Our study’s findings also raise whether just-in-time training might be useful in procedures other than infant intubations, such as central lines and chest tubes. A just-in-time model could be put into operation in two ways. For semi-urgent or non-urgent procedures, individuals could receive just-in-time training just before the procedure, as in our study. However, because many procedures can be emergent (eg, needle decompression for tension pneumothorax) with little time to perform coaching just before the procedure, an alternative would be to coach trainees briefly at the start of each shift where such procedures are likely (eg, for several minutes at the start of a shift in the intensive care unit). The key principle is to bring the time between training and implementation for a procedure much closer together. Further research is needed to assess whether just-in-timing training applies to other clinical contexts and the optimal timing and frequency of such training.

Limitations

Our study had several limitations. First, just-in-time training could slow workflow.44 However, coaching sessions each lasted a maximum of 5 to 10 minutes; were integrated into clinical workflow; and no formal complaints on operating room efficiency were received by the research coordinator, division head, or institutional review board during our study. Moreover, the airway coaches frequently had their own clinical assignments; just-in-time coaching did not impose a demanding non-clinical burden. Nonetheless, a full cost-benefit analysis should be considered.

Second, by being conducted in one center, this trial could have been subject to institutional culture. However, study participants were from 10 different training institutions, and given this trainee diversity, our results could generalize to other similar programs. Moreover, our findings should be considered as a proof of principle and suggest that additional large scale evaluation, similar to the evaluation of surgical checklists,4546 should be considered. Third, our study did not have a control placebo—that is, an instructional video or written template on intubating newborn babies. Our goal was to compare just-in-time training to our standard of care—intraoperative teaching alone by attending anesthesiologists.

Another limitation was that we did not restrict the intubating device for pragmatic reasons, because direct laryngoscopy and video laryngoscopy are used in intubating infants. We conducted a subgroup analysis of the primary outcome to account for the different modalities. Anesthesia providers strongly preferred using video laryngoscopy in this high risk population, so the 6% difference favoring the treatment group when using direct laryngoscopy did not reach significance. Furthermore, we conducted coaching sessions in a perioperative simulation suite. Many institutions might lack dedicated suites near clinical environments; however, the method could be implemented in nearby workrooms or the operating or patient room before the procedure.47 Finally, masking of participants was impossible, given the study’s nature. This source of bias was unlikely, because in both treatment and control groups, supervising attending anesthesiologists would have wanted to secure the airway in as few attempts as possible.

Conclusions

Just-in-time training with an expert coach could improve the quality of procedural care among inexperienced clinicians. In our single center, prospective randomized controlled trial, we observed increased first attempt success rates of orotracheal intubation in newborn babies and infants among trainees who received expert coaching just before intubation. Integrating a just-in-time approach into airway management training could improve patient safety and serve as a proof of concept for improving high stakes procedural outcomes more broadly. Randomized evaluation in other settings is warranted.

What is already known on this topic

  • Athletes and musicians often rehearse right before performance, just in time, with coaches who review mechanics, approach, and mental engagement to optimize outcomes

  • Limited evidence exists, especially from large, prospective randomized controlled trials, to assess whether just-in-time training with a coach could improve high risk procedural care in medicine

  • Infant intubation is a high risk procedure where minimizing intubation attempts decreases the probability of life threatening complications.

What this study adds

  • Just-in-time training by an experienced coach before infant intubation increases the first attempt success rate, decreases the mental workload, and improves competency metrics for inexperienced clinicians

  • Just-in-time training could improve the quality of high stakes procedural care more broadly

Ethics statements

Ethical approval

The institutional review board of Boston Children’s Hospital (P00034169) approved the study.

Data availability statement

Research data will be made available after publication on reasonable request after review by SGF and other study team members. A data use agreement will be required before the release of data and the institutional review board’s approval as appropriate.

Acknowledgments

We thank Jennani Durai, our department’s medical editor, for reviewing the manuscript; Robert Holzman for his scientific review of the study protocol; our colleagues, who were supportive of our research effort, accommodated our research assistants in their operating rooms, and were inspirational in their genuine devotion to the betterment of safe, compassionate anesthesia care for newborn babies and infants; and finally, the trainees who participated in the study, whose commitment to practicing and perfecting their skills for the infants under their care was inspirational.

Footnotes

  • Contributors: SGF, RSP, JMP, MLS, and PGK had full access to all the data in the study and take responsibility for data integrity and accuracy in data analysis. SGF, RSP, ABJ, SJS, GDS, PHW, JMP, MLS, and PGK conceptualized and designed the study. SGF, RSP, SJS, GDS, RSB, PHW, JMP, MLS, and PGK contributed to the methodology. SGF, RSP, SJS, SYK, JDC, IVP, KEL, SXH, JEF, JMP, MLS, and PGK contributed to acquiring, analyzing, and interpreting the data. The manuscript was drafted by SGF, RSP, ABJ, SJS, SYK, PHW, JEF, JMP, MLS, and PGK. All authors critically reviewed the manuscript for important intellectual content. SJS provided statistical analysis. SGF and PGK obtained funding. Administrative, technical, or material support was provided by SGF, SYK, JDC, IVP, KEL, SXH, RSB, and PGK. The study was supervised by SGF, RSB, and PGK. SGF is the guarantor. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

  • Funding: Funded by the Anesthesia Research Distinguished Trailblazer Award, Department of Anesthesiology, Critical Care and Pain Medicine, Boston Children’s Hospital (297600-000-847). The research conducted was independent of any involvement from the funder of the study. The funder was not involved in study design, collection, analysis, data interpretation, writing, or the decision to submit the article for publication.

  • Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/disclosure-of-interest/ and declare: support from the Department of Anesthesiology, Critical Care and Pain Medicine, Boston Children’s Hospital; no financial relationships with any organizations that might have an interest in the submitted work in the previous 3 years; and no other relationships or activities that could appear to have influenced the submitted work; PGK was a medical advisor for Verathon Medical until December 2020, outside of the submitted work; ABJ reports receiving (in the past 36 months) consulting fees unrelated to this work from Bioverativ, Merck/Sharp/Dohme, Janssen, Edwards Life Sciences, Novartis, Amgen, Eisai, Otsuka Pharmaceuticals, Vertex Pharmaceuticals, Celgene, Sanofi Aventis, Precision Health Economics, and Analysis Group; ABJ also reports receiving (in the past 36 months) income unrelated to the submitted work from hosting the podcast Freakonomics MD, and from book rights to Doubleday Books.

  • Transparency: SGF (the manuscript’s guarantor) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported, that no important aspects of the study have been omitted, and that any discrepancies are disclosed.

  • Dissemination to participants and related patient and public communities: The results of this work will be disseminated to the public through this peer reviewed publication, institutional press release, ensuing news articles, podcasts, an opinion piece authored by the study’s authors that describe the study’s findings for the public, and bedside anticipatory guidance with families as to how the study’s findings improve patient safety. The public will be involved in setting components of follow-up studies.

  • Provenance and peer review: Not commissioned; externally peer reviewed.

http://creativecommons.org/licenses/by-nc/4.0/

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

References

  1. ↵
    1. Ericsson KA
    . Acquisition and maintenance of medical expertise: a perspective from the expert-performance approach with deliberate practice. Acad Med2015;90:1471-86. doi:10.1097/ACM.0000000000000939. pmid:26375267
    OpenUrlCrossRefPubMed
  2. ↵
    1. Perretta JS,
    2. Duval-Arnould J,
    3. Poling S,
    4. et al
    . Best Practices and Theoretical Foundations for Simulation Instruction Using Rapid-Cycle Deliberate Practice. Simul Healthc2020;15:356-62. doi:10.1097/SIH.0000000000000433. pmid:32809977
    OpenUrlCrossRefPubMed
  3. ↵
    1. Hunt EA,
    2. Duval-Arnould JM,
    3. Nelson-McMillan KL,
    4. et al
    . Pediatric resident resuscitation skills improve after “rapid cycle deliberate practice” training. Resuscitation2014;85:945-51. doi:10.1016/j.resuscitation.2014.02.025. pmid:24607871
    OpenUrlCrossRefPubMed
  4. ↵
    1. Ericsson KA,
    2. Pool R
    . Peak: Secrets From the New Science of Expertise.Houghton Mifflin Harcourt, 2016.
  5. ↵
    1. Fradkin AJ,
    2. Zazryn TR,
    3. Smoliga JM
    . Effects of warming-up on physical performance: a systematic review with meta-analysis. J Strength Cond Res2010;24:140-8. doi:10.1519/JSC.0b013e3181c643a0. pmid:19996770
    OpenUrlCrossRefPubMedWeb of Science
  6. ↵
    1. Moorcroft L,
    2. Kenny DT
    . Singer and listener perception of vocal warm-up. J Voice2013;27:258.e1-13. doi:10.1016/j.jvoice.2012.12.001. pmid:23415147
    OpenUrlCrossRefPubMed
  7. ↵
    1. Moorcroft L,
    2. Kenny DT
    . Vocal warm-up produces acoustic change in singers’ vibrato rate. J Voice2012;26:667.e13-8, E18. doi:10.1016/j.jvoice.2011.10.007 pmid:22521322
    OpenUrlCrossRefPubMed
  8. ↵
    1. Aggarwal R
    . Just-in-time simulation-based training. BMJ Qual Saf2017;26:866-8. doi:10.1136/bmjqs-2017-007122. pmid:29042508
    OpenUrlFREE Full Text
  9. ↵
    1. Branzetti JB,
    2. Adedipe AA,
    3. Gittinger MJ,
    4. et al
    . Randomised controlled trial to assess the effect of a Just-in-Time training on procedural performance: a proof-of-concept study to address procedural skill decay. BMJ Qual Saf2017;26:881-91. doi:10.1136/bmjqs-2017-006656. pmid:28866621
    OpenUrlAbstract/FREE Full Text
  10. ↵
    1. Niles DE,
    2. Nishisaki A,
    3. Sutton RM,
    4. et al
    . Improved Retention of Chest Compression Psychomotor Skills With Brief “Rolling Refresher” Training. Simul Healthc2017;12:213-9. doi:10.1097/SIH.0000000000000228. pmid:28368963
    OpenUrlCrossRefPubMed
  11. ↵
    1. Braga MS,
    2. Tyler MD,
    3. Rhoads JM,
    4. et al
    . Effect of just-in-time simulation training on provider performance and patient outcomes for clinical procedures: a systematic review. BMJ Simul Technol Enhanc Learn2015;1:94-102. doi:10.1136/bmjstel-2015-000058 pmid:35515199
    OpenUrlCrossRefPubMed
  12. ↵
    1. Chen CC,
    2. Green IC,
    3. Colbert-Getz JM,
    4. et al
    . Warm-up on a simulator improves residents’ performance in laparoscopic surgery: a randomized trial. Int Urogynecol J2013;24:1615-22. doi:10.1007/s00192-013-2066-2. pmid:23575698
    OpenUrlCrossRefPubMed
  13. ↵
    1. Sun L
    . Early childhood general anaesthesia exposure and neurocognitive development. Br J Anaesth2010;105(Suppl 1):i61-8. doi:10.1093/bja/aeq302. pmid:21148656
    OpenUrlCrossRefPubMedWeb of Science
  14. ↵
    1. Gálvez JA,
    2. Acquah S,
    3. Ahumada L,
    4. et al
    . Hypoxemia, Bradycardia, and Multiple Laryngoscopy Attempts during Anesthetic Induction in Infants: A Single-center, Retrospective Study. Anesthesiology2019;131:830-9. doi:10.1097/ALN.0000000000002847. pmid:31335549
    OpenUrlCrossRefPubMed
  15. ↵
    1. Disma N,
    2. Virag K,
    3. Riva T,
    4. Kaufmann J,
    5. Engelhardt T,
    6. Habre W,
    7. NECTARINE Group of the European Society of Anaesthesiology Clinical Trial Network
    . Difficult tracheal intubation in neonates and infants. NEonate and Children audiT of Anaesthesia pRactice IN Europe (NECTARINE): a prospective European multicentre observational study. Br J Anaesth2021;126:1173-81. doi:10.1016/j.bja.2021.02.021 pmid:33812665
    OpenUrlCrossRefPubMed
  16. ↵
    1. Kane AD,
    2. Cook TM,
    3. Armstrong RA,
    4. et al
    . The incidence of potentially serious complications during non-obstetric anaesthetic practice in the United Kingdom: an analysis from the 7th National Audit Project (NAP7) activity survey. Anaesthesia2024;79:43-53. doi:10.1111/anae.16155. pmid:37944508
    OpenUrlCrossRefPubMed
  17. ↵
    1. Park R,
    2. Peyton JM,
    3. Fiadjoe JE,
    4. et al.,
    5. PeDI Collaborative Investigators,
    6. PeDI collaborative investigators
    . The efficacy of GlideScope® videolaryngoscopy compared with direct laryngoscopy in children who are difficult to intubate: an analysis from the paediatric difficult intubation registry. Br J Anaesth2017;119:984-92. doi:10.1093/bja/aex344. pmid:29028952
    OpenUrlCrossRefPubMed
  18. ↵
    1. Fiadjoe JE,
    2. Nishisaki A,
    3. Jagannathan N,
    4. et al
    . Airway management complications in children with difficult tracheal intubation from the Pediatric Difficult Intubation (PeDI) registry: a prospective cohort analysis. Lancet Respir Med2016;4:37-48. doi:10.1016/S2213-2600(15)00508-1. pmid:26705976
    OpenUrlCrossRefPubMed
  19. ↵
    1. Patel R,
    2. Lenczyk M,
    3. Hannallah RS,
    4. McGill WA
    . Age and the onset of desaturation in apnoeic children. Can J Anaesth1994;41:771-4. doi:10.1007/BF03011582. pmid:7954992
    OpenUrlCrossRefPubMedWeb of Science
  20. ↵
    1. Byrne AJ,
    2. Oliver M,
    3. Bodger O,
    4. et al
    . Novel method of measuring the mental workload of anaesthetists during clinical practice. Br J Anaesth2010;105:767-71. doi:10.1093/bja/aeq240. pmid:20846966
    OpenUrlCrossRefPubMedWeb of Science
  21. ↵
    1. Garcia-Marcinkiewicz AG,
    2. Kovatsis PG,
    3. Hunyady AI,
    4. et al.,
    5. PeDI Collaborative investigators
    . First-attempt success rate of video laryngoscopy in small infants (VISI): a multicentre, randomised controlled trial. Lancet2020;396:1905-13. doi:10.1016/S0140-6736(20)32532-0. pmid:33308472
    OpenUrlCrossRefPubMed
  22. ↵
    1. Hodgson KA,
    2. Owen LS,
    3. Kamlin COF,
    4. et al
    . Nasal High-Flow Therapy during Neonatal Endotracheal Intubation. N Engl J Med2022;386:1627-37. doi:10.1056/NEJMoa2116735. pmid:35476651
    OpenUrlCrossRefPubMed
  23. ↵
    1. Hancock P,
    2. Meshkati N
    1. Hart S,
    2. Staveland L
    . Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. In: Hancock P, Meshkati N, eds. Human Mental Workload.North Holland Press, 1988: 139-83doi:10.1016/S0166-4115(08)62386-9.
    OpenUrlCrossRef
  24. ↵
    1. Mazur LM,
    2. Mosaly PR,
    3. Hoyle LM,
    4. Jones EL,
    5. Marks LB
    . Subjective and objective quantification of physician’s workload and performance during radiation therapy planning tasks. Pract Radiat Oncol2013;3:e171-7. doi:10.1016/j.prro.2013.01.001. pmid:24674415
    OpenUrlCrossRefPubMed
  25. ↵
    1. Yurko YY,
    2. Scerbo MW,
    3. Prabhu AS,
    4. Acker CE,
    5. Stefanidis D
    . Higher mental workload is associated with poorer laparoscopic performance as measured by the NASA-TLX tool. Simul Healthc2010;5:267-71. doi:10.1097/SIH.0b013e3181e3f329. pmid:21330808
    OpenUrlCrossRefPubMed
  26. ↵
    1. Prottengeier J,
    2. Petzoldt M,
    3. Jess N,
    4. et al
    . The effect of a standardised source of divided attention in airway management: A randomised, crossover, interventional manikin study. Eur J Anaesthesiol2016;33:195-203. doi:10.1097/EJA.0000000000000315. pmid:26656770
    OpenUrlCrossRefPubMed
  27. ↵
    1. Ambardekar AP,
    2. Rosero EB,
    3. Bhoja R,
    4. et al
    . A Randomized Controlled Trial Comparing Learners’ Decision-making, Anxiety, and Task Load During a Simulated Airway Crisis Using Two Difficult Airway Aids. Simul Healthc2019;14:96-103. doi:10.1097/SIH.0000000000000362. pmid:30908420
    OpenUrlCrossRefPubMed
  28. ↵
    1. Yu D,
    2. Lowndes B,
    3. Thiels C,
    4. et al
    . Quantifying Intraoperative Workloads Across the Surgical Team Roles: Room for Better Balance?World J Surg2016;40:1565-74. doi:10.1007/s00268-016-3449-6. pmid:26952115
    OpenUrlCrossRefPubMed
  29. ↵
    1. Kennedy-Metz LR,
    2. Wolfe HL,
    3. Dias RD,
    4. Yule SJ,
    5. Zenati MA
    . Surgery Task Load Index in Cardiac Surgery: Measuring Cognitive Load Among Teams. Surg Innov2020;27:602-7. doi:10.1177/1553350620934931. pmid:32938323
    OpenUrlCrossRefPubMed
  30. ↵
    1. Hertzum M
    . Reference values and subscale patterns for the task load index (TLX): a meta-analytic review. Ergonomics2021;64:869-78. doi:10.1080/00140139.2021.1876927. pmid:33463402
    OpenUrlCrossRefPubMed
  31. ↵
    1. Cormack RS,
    2. Lehane J
    . Difficult tracheal intubation in obstetrics. Anaesthesia1984;39:1105-11. doi:10.1111/j.1365-2044.1984.tb08932.x pmid:6507827
    OpenUrlCrossRefPubMedWeb of Science
  32. ↵
    1. Yentis SM,
    2. Lee DJ
    . Evaluation of an improved scoring system for the grading of direct laryngoscopy. Anaesthesia1998;53:1041-4. doi:10.1046/j.1365-2044.1998.00605.x. pmid:10023271
    OpenUrlCrossRefPubMedWeb of Science
  33. ↵
    1. Park RS,
    2. Rattana-Arpa S,
    3. Peyton JM,
    4. et al
    . Risk of Hypoxemia by Induction Technique Among Infants and Neonates Undergoing Pyloromyotomy. Anesth Analg2021;132:367-73. doi:10.1213/ANE.0000000000004344. pmid:31361669
    OpenUrlCrossRefPubMed
  34. ↵
    1. Disma N,
    2. Asai T,
    3. Cools E,
    4. et al.,
    5. and airway guidelines groups of the European Society of Anaesthesiology and Intensive Care (ESAIC) and the British Journal of Anaesthesia (BJA)
    . Airway management in neonates and infants: European Society of Anaesthesiology and Intensive Care and British Journal of Anaesthesia joint guidelines. Eur J Anaesthesiol2024;41:3-23. doi:10.1097/EJA.0000000000001928. pmid:38018248
    OpenUrlCrossRefPubMed
  35. ↵
    1. O’Shea JE,
    2. Thio M,
    3. Kamlin CO,
    4. et al
    . Videolaryngoscopy to Teach Neonatal Intubation: A Randomized Trial. Pediatrics2015;136:912-9. doi:10.1542/peds.2015-1028. pmid:26482669
    OpenUrlCrossRefPubMed
  36. ↵
    1. Peyton J,
    2. Park RS,
    3. Flynn S,
    4. Kovatsis P
    . In the Director’s Chair: Screen Time, Coaching, and Intubation in the PICU. Crit Care Med2023;51:981-3. doi:10.1097/CCM.0000000000005875. pmid:37318296
    OpenUrlCrossRefPubMed
  37. ↵
    1. Hu YY,
    2. Mazer LM,
    3. Yule SJ,
    4. et al
    . Complementing Operating Room Teaching With Video-Based Coaching. JAMA Surg2017;152:318-25. doi:10.1001/jamasurg.2016.4619. pmid:27973648
    OpenUrlCrossRefPubMed
  38. ↵
    1. Nishisaki A,
    2. Donoghue AJ,
    3. Colborn S,
    4. et al
    . Effect of just-in-time simulation training on tracheal intubation procedure safety in the pediatric intensive care unit. Anesthesiology2010;113:214-23. doi:10.1097/ALN.0b013e3181e19bf2. pmid:20526179
    OpenUrlCrossRefPubMedWeb of Science
  39. ↵
    1. Kessler D,
    2. Pusic M,
    3. Chang TP,
    4. et al.,
    5. INSPIRE LP investigators
    . Impact of Just-in-Time and Just-in-Place Simulation on Intern Success With Infant Lumbar Puncture. Pediatrics2015;135:e1237-46. doi:10.1542/peds.2014-1911. pmid:25869377
    OpenUrlCrossRefPubMed
  40. ↵
    1. Kamdar G,
    2. Kessler DO,
    3. Tilt L,
    4. et al
    . Qualitative evaluation of just-in-time simulation-based learning: the learners’ perspective. Simul Healthc2013;8:43-8. doi:10.1097/SIH.0b013e31827861e8. pmid:23299050
    OpenUrlCrossRefPubMed
  41. ↵
    1. Gizicki E,
    2. Assaad MA,
    3. Massé É,
    4. Bélanger S,
    5. Olivier F,
    6. Moussa A
    . Just-In-Time Neonatal Endotracheal Intubation Simulation Training: A Randomized Controlled Trial. J Pediatr2023;261:113576. doi:10.1016/j.jpeds.2023.113576. pmid:37353151
    OpenUrlCrossRefPubMed
  42. ↵
    1. Birkmeyer JD,
    2. Stukel TA,
    3. Siewers AE,
    4. Goodney PP,
    5. Wennberg DE,
    6. Lucas FL
    . Surgeon volume and operative mortality in the United States. N Engl J Med2003;349:2117-27. doi:10.1056/NEJMsa035205. pmid:14645640
    OpenUrlCrossRefPubMedWeb of Science
  43. ↵
    1. Hockenberry JM,
    2. Helmchen LA
    . The nature of surgeon human capital depreciation. J Health Econ2014;37:70-80. doi:10.1016/j.jhealeco.2014.06.001. pmid:24973949
    OpenUrlCrossRefPubMed
  44. ↵
    1. Greig PR,
    2. Higham HE,
    3. Darbyshire JL,
    4. Vincent C
    . Go/no-go decision in anaesthesia: wide variation in risk tolerance amongst anaesthetists. Br J Anaesth2017;118:740-6. doi:10.1093/bja/aew444. pmid:28510736
    OpenUrlCrossRefPubMed
  45. ↵
    1. Pronovost P,
    2. Needham D,
    3. Berenholtz S,
    4. et al
    . An intervention to decrease catheter-related bloodstream infections in the ICU. N Engl J Med2006;355:2725-32. doi:10.1056/NEJMoa061115. pmid:17192537
    OpenUrlCrossRefPubMedWeb of Science
  46. ↵
    1. Haynes AB,
    2. Weiser TG,
    3. Berry WR,
    4. et al.,
    5. Safe Surgery Saves Lives Study Group
    . A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med2009;360:491-9. doi:10.1056/NEJMsa0810119. pmid:19144931
    OpenUrlCrossRefPubMedWeb of Science
  47. ↵
    1. Weinstock PH,
    2. Kappus LJ,
    3. Garden A,
    4. Burns JP
    . Simulation at the point of care: reduced-cost, in situ training via a mobile cart. Pediatr Crit Care Med2009;10:176-81. doi:10.1097/PCC.0b013e3181956c6f. pmid:19188878
    OpenUrlCrossRefPubMed
View Abstract
Back to top