|
|
||||||||
Guest Access | Sign In via User Name/Password |
|||||||||
* From the Department of Medicine (Drs. Wayne, Didwania, Feinglass, and Barsuk, and Ms. Fudala), and the Office of Medical Education and Faculty Development (Dr. McGaghie), Northwestern University Feinberg School of Medicine, Chicago, IL.
Correspondence to: Diane B. Wayne, MD, Northwestern University, Department of Medicine, 251 E Huron St, Galter 3-150, Chicago, IL 60611; e-mail: dwayne{at}northwestern.edu
Abstract
Background: Simulation technology is widely used in medical education. Linking educational outcomes achieved in a controlled environment to patient care improvement is a constant challenge.
Methods: This was a retrospective case-control study of cardiac arrest team responses from January to June 2004 at a university-affiliated internal medicine residency program. Medical records of advanced cardiac life support (ACLS) events were reviewed to assess adherence to ACLS response quality indicators based on American Heart Association (AHA) guidelines. All residents received traditional ACLS education. Second-year residents (simulator-trained group) also attended an educational program featuring the deliberate practice of ACLS scenarios using a human patient simulator. Third-year residents (traditionally trained group) were not trained on the simulator. During the study period, both simulator-trained and traditionally trained residents responded to ACLS events. We evaluated the effects of simulation training on the quality of the ACLS care provided.
Results: Simulator-trained residents showed significantly higher adherence to AHA standards (mean correct responses, 68%; SD, 20%) vs traditionally trained residents (mean correct responses, 44%; SD, 20%; p = 0.001). The odds ratio for an adherent ACLS response was 7.1 (95% confidence interval, 1.8 to 28.6) for simulator-trained residents compared to traditionally trained residents after controlling for patient age, ventilator, and telemetry status.
Conclusions: A simulation-based educational program significantly improved the quality of care provided by residents during actual ACLS events. There is a growing body of evidence indicating that simulation can be a useful adjunct to traditional methods of procedural training.
Key Words: advanced cardiac life support cardiopulmonary resuscitation medical education simulation technology
The use of simulation technology has great potential to shape medical education, certification, licensure, and the quality of care.1 Simulation has demonstrated its effectiveness to achieve, measure, and maintain trainee skill in the performance of a variety of clinical procedures including laparoscopic surgery,234 endoscopy,5 advanced cardiac life support (ACLS),678 emergency airway management,9 trauma resuscitation,1011 bronchoscopy,12 and carotid angiography.13 Although this technology has much promise, a key challenge is to link performance in the controlled simulation environment to the quality of the patient care delivered. Small studies have evaluated clinical performance after simulator training with encouraging results, but their scope is limited.9141516171819
In July 2003, we initiated a simulation-based ACLS training program for internal medicine residents featuring 8 h of deliberate practice on a full-body patient simulator, immediate feedback, and rigorous outcome measures. We evaluated training outcomes using a randomized trial with a waiting list control group. Results showed that residents ACLS skills increased 38%6, were maintained over a period of 14 months,8 and that the training was rated highly as an adjunct to traditional didactic and clinical experience. The present study evaluates whether simulation-based ACLS education affects the quality of care during actual ACLS events at our institution.
Materials and Methods
Design
This was a retrospective case-control study20 of cardiac arrest team responses at Northwestern Memorial Hospital (NMH) from January to June 2004. An educational program had been started 6 months earlier for second-year internal medicine residents featuring simulation-based training in ACLS. A 6-month study period was selected because it provided an opportunity to compare two groups of ACLS team leaders as a natural quasi-experiment.21 Cases and controls are team responses to ACLS events divided into those who were directed by simulator-trained leaders vs those who were managed by traditionally trained leaders. The Northwestern University institutional review board approved the study and waived informed consent.
Setting
NMH is a tertiary health-care facility located in urban Chicago and is the principal setting for graduate medical education under the auspices of Northwestern University. At NMH, cardiac arrest teams, led by an on-call second-year or third-year internal medicine resident, respond to all "code blue" calls. All residents completed an American Heart Association (AHA) ACLS provider course at the beginning of the first year of training and again at the start of the third year of training (Fig 1
).
|
The course faculty developed case studies and management checklists for the six most common ACLS scenarios encountered at NMH. These were asystole, ventricular fibrillation, supraventricular tachycardia, ventricular tachycardia, symptomatic bradycardia, and pulseless electrical activity. Educational content and assessment tools were based entirely on the ACLS Provider Manual22 used by the AHA as instructional materials for ACLS provider courses. Teaching sessions allowed groups of two to four residents time to practice the protocols and procedures, and to receive standardized education from simulator faculty. Debriefing allowed the residents to ask questions, review algorithms, and receive feedback. The four teaching sessions were presented in uniform order as follows: (1) procedures (intubation, central line placement, pericardiocentesis, and needle decompression of tension pneumothorax); (2) pulseless arrhythmias (asystole, ventricular fibrillation, and pulseless electrical activity); (3) tachycardias (supraventricular and ventricular); and (4) bradycardias (second-degree and third-degree atrioventricular block).
Third-year residents (ie, the traditionally trained group [n = 40]) had more postgraduate training and had recently renewed their ACLS provider status through an AHA ACLS course, as shown in Figure 1, but were not trained using the simulator. The study period provided a unique opportunity for a natural quasi-experiment21 to compare these two groups.
Data Collection
A log of cardiac arrest team responses is maintained by the NMH Cardiopulmonary Resuscitation Quality Committee. This log was used to identify the 72 cardiac arrest team responses during the 6-month study period. All 72 medical records were available for review. Twenty-two medical records were excluded because they were respiratory arrests primarily managed by anesthesiology or were non-ACLS situations such as near syncope, chest pain, or asymptomatic hypotension. Two records were excluded because the cardiac arrest flowsheet could not be obtained.
A sequential review of the remaining 48 medical records was conducted. Data were abstracted from the cardiac arrest flowsheet completed at each ACLS event by an ICU nurse trained and assigned by the Cardiopulmonary Resuscitation Quality Committee. Because the primary study reviewers were not blinded to residents training status, 20 randomly selected records were re-reviewed by a senior hospitalist faculty member who was blinded to residents training status and the results of the first review. Performance ratings from the second reviewer were compared with ratings by the primary reviewers to assess interrater reliability.
Outcome Measure
Mirroring the simulation training evaluations,6 medical abstraction was based on the 2003–2004 AHA guidelines, which list four key steps across six common ACLS events (ie, asystole, ventricular fibrillation, supraventricular tachycardia, ventricular tachycardia, symptomatic bradycardia, and pulseless electrical activity) to compute the total correct performance score as a percentage. The four measures were selected because they were common across ACLS scenarios, did not depend on physician judgment, could be scored objectively, and were based on algorithms from the AHA ACLS textbook.22 The four key steps are as follows:
For the most common condition, refractory ventricular fibrillation (n = 16), a more detailed assessment was possible due to the AHA algorithm for this scenario.22 The algorithm consisted of the following: (1) use of defibrillation immediately after administering basic life support; (2) administration of three shocks without interruption; (3) use of a drug-shock-drug sequence; and (4) administration of an antiarrhythmic drug at the appropriate point in the algorithm. Abstracted data also included patient age and pre-cardiac arrest ventilator and telemetry status.
Adherent responses to each ACLS event were defined as compliance with AHA guidelines of
75%, based on adding all possible correctly performed performance measures. This cutoff was used because it is identical to the mean of the passing performance standards set by an expert panel in an earlier ACLS standard-setting study.23
Statistical Analysis
Interrater reliability was assessed using the
coefficient24 adjusted using the formula of Brennan and Prediger.25 Group differences between simulator-trained and traditionally trained residents were evaluated using the t test and the
2 test. Logistic regression was computed to determine the odds ratio of an adherent ACLS response, testing the difference between groups controlling for patient age, and preevent ventilator and telemetry status (each coded "Y"/"N"). Logistic regression variables were selected because they were documented in the medical record and had been used in prior studies2627 of survival from in-hospital cardiac arrest events.
Results
The mean interrater reliability for the 20 randomly selected ACLS events across the eight performance measures was high (
= 0.87). This indicates that there was very little disagreement about abstraction of ACLS performance data.
Twenty ACLS events were led by simulator-trained second-year residents and 28 were led by traditionally trained third-year residents. As shown in Table 1 , no difference between patients in either group was shown by age, ventilator, or telemetry status.
|
0.001) [Fig 2
].
|
75% cut point for an adherent performance, 13 of 20 ACLS responses (65%) led by simulator-trained residents achieved this standard vs 6 of 28 (21.4%) responses led by traditionally trained residents (
2[1] = 7.7; p
0.01). Simulator-trained residents were over seven times more likely to lead an adherent ACLS response than traditionally trained residents (odds ratio, 7.1; 95% confidence interval, 1.8 to 28.6) after controlling for patient age, ventilator, and telemetry status in logistic regression. Twenty-two of the 48 patients survived the ACLS event, and 3 patients survived to hospital discharge. Nine of the 20 patients (45%) managed by simulator-trained residents survived the event and 13 of the 28 patients (46.4%) managed by traditionally trained residents survived the event (p = 0.92). Although postevent survival was not different between groups, the mean survival time to death or hospital discharge among the nine surviving patients managed by simulator-trained residents was 194.7 h compared to 107.1 h for the 13 surviving patients managed by traditionally trained residents (p = 0.11). Two of the patients surviving to hospital discharge were in the simulator-trained group (discharge rate, 10%), and one patient was in the traditionally trained group (eventual discharge rate, 3.6%; p = 0.36).
Discussion
This study amplifies the outcomes of past research on simulation-based ACLS medical education by extending those research findings to the quality of care provided during real ACLS events. Our prior work documented the ability of simulation-based training to significantly increase residents ACLS skills,6 to allow for the setting of mastery standards,7 and to demonstrate the retention of skill over a 14-month study period.8 The research situation described in this article presented a unique opportunity to assess clinical responses of simulator-trained vs traditionally trained residents in an actual patient care environment. Our study effectively demonstrates the ability of a simulation-based education program to improve adherence to AHA ACLS guidelines in a group of residents, even though they had less clinical experience and less traditional ACLS training (one vs two provider courses) than the comparison group (Fig 1).
While supporting a role for simulation training grounded in deliberate practice, our results highlight an unanswered question, "Can competence be evaluated independent of outcomes?"28 The clinical process was fulfilled by the ACLS training and evaluation program, but differences in clinical outcome were not achieved because most patients died regardless of the treatment they received. Our survival to hospital discharge rate of 3 of 48 patients is within the range of other published studies29 and reflects the high prearrest morbidity of this patient population. Many factors may also contribute to survival from in-hospital cardiopulmonary arrest including patient characteristics, initial cardiac rhythm, and circadian variation.2930 Given these cofounders and our small sample size, it is not surprising that our results did not conclusively show improved survival as a result of simulator training. Whether this would be seen in a larger patient sample is unknown, although a trend toward longer postevent survival and an improved survival to hospital discharge rate were seen.
Many of the ACLS events led by our residents did not meet the minimum standard for resuscitation despite prior conventional training and experience. This finding is consistent with those of other studies that show clinical experience alone is often insufficient to ensure the acquisition of such basic clinical skills as chest radiograph interpretation,31 hypertension diagnosis and management,32 cardiac auscultation,33 and emergency airway management.34 Poor adherence to AHA guidelines by well-trained hospital staff during in-hospital ACLS events3536 and a rapid decline of skill after traditional ACLS education37 have also been well documented. Our results show a dramatic improvement in adherence to accepted guidelines in the clinical setting among residents who had an opportunity for focused, deliberate practice in a controlled simulation environment. Our prior observations8 of long-term retention of these skills following simulator training are encouraging and warrant further study.
Our study has several limitations. It represents a small sample of events at a single institution over a short time period. It used a case-control retrospective design rather than a prospective randomized design. The possibility of omissions or errors in chart documentation cannot be excluded. Clinical factors other than the training status of team leaders, such as input or cuing from other members of the resuscitation team, may influence adherence to ACLS guidelines. The existence of other confounding or uncontrolled factors cannot be definitively ruled out. However, the magnitude and consistency of the statistical results lead us to believe that the skill improvements gained in the simulated environment678 can be generalized to the clinical setting and are not an artifact of chance or other variables.
In conclusion, the use of a simulation-based training program improved the quality of resuscitation efforts by internal medicine residents in actual ACLS events at our institution. We believe traditional bedside and clinical teaching in medical education should be amplified to include simulation-based training. This may be especially useful in high-acuity skills such as ACLS, which are required for board certification38 yet occur infrequently.27 In patients with critical illness, teams of physicians need to respond effectively to provide high-quality patient care. This study adds to the growing body of knowledge that simulation-based education improves the procedural skills of physicians234567891011121314151617181934 and may enhance the quality of their patient care.914151617181934 In this study, deliberate practice is shown to be a powerful tool to boost the competence of physicians and the quality of their patient care in actual ACLS events.
Acknowledgements
We thank the Northwestern University internal medicine residents for their dedication to patient care and education. We acknowledge Charles Watts, MD, and J. Larry Jameson, MD, PhD, for their support and encouragement of this work. We appreciate the input of anonymous reviewers who helped to strengthen this article.
Footnotes
Abbreviations: ACLS = advanced cardiac life support; AHA = American Heart Association; NMH = Northwestern Memorial Hospital
The study was supported by the Excellence in Academic Medicine Act of the Illinois Department of Public Aid administered through Northwestern Memorial Hospital.
The authors have reported to the ACCP that no significant conflicts of interest exist with any companies/organizations whose products or services may be discussed in this article.
Received for publication January 15, 2006. Accepted for publication April 9, 2007.
References
Related Editorial
This article has been cited by other articles:
![]() |
W. Dunn and J. G. Murphy Simulation: About Safety, Not Fantasy Chest, January 1, 2008; 133(1): 6 - 9. [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |