|
|
||||||||
Guest Access | Sign In via User Name/Password |
|||||||||

* Dr. Cook is a Research Chair of the Canadian Institutes of Health Research.
Dr. Grimshaw holds a Canada Research Chair in Health Knowledge Transfer and Uptake.
Correspondence to: Holger J. Schünemann, MD, PhD, FCCP, Department of Medicine, University at Buffalo, ECMCCC142, 462 Grider St, Buffalo, NY 14215; e-mail: HJS{at}buffalo.edu
| Abstract |
|---|
|
|
|---|
Key Words: evidence-based medicine implementation practice guidelines
| Introduction |
|---|
|
|
|---|
In theory, dissemination of evidence-based guidelines derived from systematic reviews should result in more informed decision making and improved patient outcomes. Despite their widespread dissemination, however, the application of the American College of Chest Physicians (ACCP) antithrombotic guidelines in practice remains incomplete. For example, Arnold et al3 demonstrated that clinicians could have prevented 17.4% of 253 episodes of venous thromboembolism (VTE) if they had applied adequate prophylaxis as recommended in the guidelines. Only 8.3% of episodes occurred in spite of prophylaxis according to the guidelines. Other observational studies456 of the most seriously ill hospitalized patients in the ICU have shown a variable rate of VTE prophylaxis ranging as low as 33%, even after excluding patients with contraindications to heparin. A recent survey,7 the National Anticoagulation Benchmark and Outcomes Report, showed that 30% of patients with atrial fibrillation at moderate risk for cardioembolic stroke who were eligible for aspirin or vitamin K antagonists did not receive either of these highly efficacious therapies.
These studies illustrate the gap that exists between evidence and its application in practice. Clearly, the development and dissemination of practice guidelines alone have an insufficient effect on altering clinician behavior and practice patterns. Obvious explanations for why clinicians do not consistently apply the ACCP guidelines in practice are that the evidence supporting ACCP recommendations may not be sufficiently strong, or the attitudes, values, and preferences underlying ACCP recommendations may differ from those of physicians and their patients. In the ACCP terminology of grading recommendations, this may be true for Grade 2, and particularly Grade 2C recommendations, but it does not explain practice inconsistent with Grade 1, and particularly 1A recommendations (aspirin or vitamin K antagonist for the prevention of cardioembolic stroke in patients with atrial fibrillation at moderate stroke risk, for instance).
Roadblocks encountered in moving from evidence to action may help to explain the gap between recommendations, in particular Grade 1A recommendations, and practice.8 Authorities have described barriers to evidence uptake, and suggested a number of strategies to overcome them.9 The barriers include physicians lack of awareness of guidelines, awareness of the guidelines but lack of detailed knowledge about them, lack of time to consider the guidelines, negative attitudes about guidelines in general, fear of jeopardizing the patient-clinician relationship, and perceived lack of support by peers and influential administrators.101112 Other barriers are forgetfulness, lack of forcing strategies, time pressures, lack of equipment or appropriate systems, inappropriate skill mix, and absence of care processes that promote guideline implementation.
Two resources exist to inform strategies for overcoming these barriers. First, one could emulate the methods of the pharmaceutical industry that successfully changes clinician behavior. We will not, however, further discuss the strategies employed by the pharmaceutical industry with its resources that outweigh resources of most organizations or agencies.13 Second, one might conduct or utilize a systematic review of randomized controlled trial (RCT) evidence comparing different strategies of changing clinician behavior. In this area of health-care research, as in many others, effects are likely to be small to modest, the understanding of confounders is limited, and the consequences of false answers may result in financial debacles. Therefore, the arguments for this approach are congruent with the arguments for an evidence-based approach focusing on systematic reviews as basis for decision making. A systematic review should focus on systematically summarizing and analyzing the highest quality evidence to improve precision and to explore consistency and generalizability of effects in different contexts. We have adopted the latter approach for this article relying on a modification of the methods reported by Grimshaw et al.14
We applied the grading system described elsewhere in this Supplement15 to grade the methodologic quality of the primary studies described in systematic reviews, and the strength of recommendations for successful guideline implementation. The format of this chapter differs slightly from other chapters in this Supplement.16
| IDENTIFYING EVIDENCE FOR IMPLEMENTATION STRATEGIES |
|---|
|
|
|---|
|
|
| SURVEY OF ACCP CONFERENCE PARTICIPANTS |
|---|
|
|
|---|
For this survey, we defined feasibility as "feasibility of implementing the described intervention in your practice setting when you consider the organizational structure and setup," cost as "cost associated with introducing the implementation strategy in your practice setting," and acceptability as "likelihood that colleagues in your practice setting would adopt, utilize and/or follow this strategy if it were implemented." Building on experience with a pilot survey in the United Kingdom,14 we presented statements regarding feasibility, acceptability, and cost, asking respondents to rate the extent to which they agreed with the statements on a 7-point Likert-type scale ranging from strongly disagree to strongly agree.
For example, the statement about computerized reminders in this survey included the following: please consider computer reminders. A reminder is a support mechanism that appears before an action has been completed. Examples of computer reminders include the following: (1) computer-generated reports sent annually to clinicians by a central administrative system; and (2) in the context of a computerized tracking and/or electronic record system, computer-generated "alerts" and "messages" to clinicians, derived from management guidelines. Alerts would be sent to clinicians every time an event occurs in a patient (eg, atrial fibrillation develops). A message would be a prompt to the clinician to commence appropriate management when the clinician opens that patients electronic record (eg, a reminder that the patient should be considered for antithrombotic therapy).
We then asked the participants to think about the definitions of feasibility, cost, and acceptability for computer reminders in their health-care setting, and indicate their level of agreement with the three statements below. The three questions about feasibility, acceptability, and cost we asked are as follows: (1) How feasible are computer reminders to improve evidence implementation in your health-care setting? We consider a feasible intervention as one that is easy to implement without major logistical burden, and an infeasible intervention as one that is associated with major logistical burden in your practice setting. Respondents provided answers on a scale ranging from 1 (infeasible) to 7 (feasible). (2) How acceptable are computer reminders in your health-care setting? We consider an acceptable intervention as one that colleagues would welcome, and an unacceptable intervention as one that would upset them if it were implemented in your practice setting. Respondents provided answers on a scale ranging from 1 (unacceptable) to 7 (acceptable). (3) How do you evaluate cost related to computer reminders in your health-care setting? We consider cost as cost associated with introducing the implementation strategy in your practice setting. Respondents provided answers on a scale ranging from 1 (very high cost) to 7 (very low cost).
We calculated mean scores and SDs around the mean score and conducted a multivariate analysis to compare the mean scores between implementation strategies for each of the three questions. Because of the multiple statistical tests we performed, we considered p < 0.01 as statistically significant.
Of 87 ACCP Conference participants, 67 participants (77% response rate) completed the survey. Table 3 shows the mean scores for each of the three questions about feasibility, acceptability, and cost for each of the implementation strategies in the survey. Respondents indicated that distribution of educational material and educational meetings were feasible, acceptable, and of low cost; differences between these two strategies were not statistically significant. Respondents reported educational outreach visits, computer reminders, and patient-mediated interventions as feasible and acceptable, but rated educational outreach visits as more costly. Audit and feedback scored lower than all other implementation strategies in terms of feasibility and acceptability; respondents also rated audit and feedback as unfavorably as educational outreach visits in terms of cost.
|
| STUDIES INVESTIGATING GUIDELINE IMPLEMENTATION STRATEGIES IN ANTITHROMBOTIC AND THROMBOLYTIC THERAPY |
|---|
|
|
|---|
We used the EPOC database to obtain evidence about effectiveness of implementation strategies focusing on guidelines for antithrombotic or thrombolytic interventions (http://www.epoc.uottawa.ca/register.htm). We reviewed the "titles" and the "clinical problem field" of the EPOC registry to identify all studies that referred to antithrombotic or thrombolytic interventions in duplicate until the end of the year 2000. Studies had to include a control group and include one of the implementation strategies described above. If the reference included one of the strategies or the title did not mention the specific strategy, we reviewed the abstract of the study. We identified 23 relevant studies.1718192021222324252627282930313233343536373839 The studies varied widely in methodology and clinical problems. For example, Anderson et al18 determined the effect of audit and feedback on physician practices in the prevention of venous thromboembolism. Fifteen short-stay hospitals were randomly assigned to group audit and feedback or group-plus-individual feedback or to a control group that received no intervention. There was no significant difference in the use of prophylaxis in hospitals whose physicians received individual audit and feedback compared with hospitals whose physicians received group interventions alone (identical increases of 28% in prophylaxis use over a period of 3 years). They noted that individual physician audit and feedback appeared to provide no additional benefit and that, in general, prophylaxis for VTE remained underutilized.
Another example provides the study by Durieux et al,22 who used a time series design to evaluate whether a computer-reminder model for presentation of guidelines about VTE prevention improves appropriate prophylaxis use. They evaluated 1,971 patients who underwent orthopedic surgery after integrating a computer-reminder system into daily medical practice during three 10-week intervention periods, alternated with four 10-week control periods, with a 4-week washout between each period. They found that physicians complied with guidelines in 82.8% (95% confidence interval [CI], 77.6 to 87.1%) of cases during control periods and in 94.9% (95% CI, 92.5 to 96.6%) of cases during intervention periods. During each intervention period, the appropriateness of prescription increased significantly. Each time the computer reminders were removed, physician practice reverted to that observed before initiation of the intervention. The relative risk of inappropriate practice decisions during control periods vs intervention periods was 3.8 (95% CI, 2.7 to 5.4). The investigators concluded that implementation of clinical guidelines for VTE prophylaxis through a computer-reminder system in an orthopedic surgery department and integrated into the hospital information system changed physician behavior and improved compliance with guidelines.
1.0 Guideline Implementation Strategies Investigated in RCTs
1.1. Distribution of educational material
Seven RCTs40414243444546 investigated the distribution of educational material, predominantly in the primary care setting (eg, adherence to chest radiograph-ordering guidelines after guideline distribution). The quality of these RCTs was poor. Only one RCT40 reported allocation concealment, three RCTs404243 reported complete follow-up, and three RCTs404243 reported blinded outcome assessment. No RCT used guidelines in the antithrombotic or thrombolytic field. The effectiveness of distributing educational material on process-of-care outcomes was modest across these seven trials. The studies did not report effects on patient outcomes. The overall quality of these studies was low, but this implementation strategy received the highest ratings in our survey. Our survey indicated that distribution of educational material is generally feasible, acceptable, and of relatively low cost.
Recommendation
1.1. We recommend that appreciable resources be devoted to distribution of educational material (Grade 2B).
1.2 Educational meetings
We identified three RCTs404748 investigating the effect of educational meetings on guideline uptake. There was no RCT related to antithrombotic or thrombolytic therapy. Two studies4748 investigated educational meetings across different guidelines such as end-of-life care and asthma management. Only one study40 concealed allocation, all studies had complete follow-up, and one study48 reported blinded outcome assessment. There was no consistent effect across these three studies on process of care. Only one study47 reported improvement of process of care, but no effect on patient outcomes. In our survey, ACCP Conference participants rated educational meetings generally as feasible, acceptable, and relatively low cost. Overall, the results of the available studies are inconsistent, and few data on patient important outcomes are available for an implementation strategy that might be generally feasible, acceptable, and relatively inexpensive.
Recommendation
1.2. We suggest that few resources be devoted to educational meetings (Grade 2B).
1.3. Educational outreach visits
Three RCTs495051 investigated educational outreach visits across different disciplines. Studies50 dealt with issues such as detailing by pharmacists on hyperlipidemia management, but there was no study in the area of antithrombotic or thrombolytic therapy area. Two studies4950 reported allocation concealment, no study reported complete follow-up, and one study49 reported blinded outcome assessment. The studies reported small effects on process of care that were not statistically significant, but the studies did not measure patient outcomes. The survey indicated moderate acceptability and feasibility, but unfavorable cost ratings. Thus, for educational outreach visits there are limited data, small effects, and uncertain feasibility, acceptability, and relatively high cost
Recommendation
1.3. We suggest that few resources be devoted to educational outreach visits (Grade 2B).
1.4. Computer reminders
Twelve RCTs525354555657585960616263 fulfilled the inclusion criteria and explored the effects of computerized reminders. These RCTs predominantly evaluated the effects of preventive care measures or appropriate medication prescription. For example, one study61 evaluated the effect of computer reminders on following adequate prescription of initial antihypertensive therapy. No RCT focused on antithrombotic or thrombolytic therapy. Only three studies535961 reported allocation concealment, one study54 reported complete follow-up, and two studies525961 reported blinded outcome assessment. Nine studies reported improvement in the process of care with modest-to-moderate absolute improvements. No study reported assessment of patient outcomes. The survey indicated that computer reminders may have limited feasibility, acceptability, and relatively high cost. Thus, there was high-quality evidence for a modest-to-moderate efficacious implementation strategy for which cost, feasibility, and acceptability may be prohibitive in some practice settings.
Recommendation
1.4. We suggest that appreciable resources be devoted to computer reminders (Grade 2A).
1.5. Patient-mediated interventions
Four RCTs59646566 involved patient-mediated interventions to increase guideline uptake; one of the studies included two comparisons. All studies took place in primary care settings. For example, in one study65 investigators randomized eligible patients to receive information by letter about mammography screening for breast cancer. Only one study59 reported concealment of allocation, no study reported complete follow-up, and two studies6466 reported blinded outcome assessment. Three of four comparisons that measured process of care reported improvement. The two RCTs assessing patient outcomes reported improvement. In general, the beneficial effects of patient-mediated interventions were moderate, but there was inconsistency in the reported effects across studies. Our survey indicated that patient-mediated interventions may be feasible, acceptable, and associated with relatively low cost. However, enthusiasm is limited because of unit of analysis errors in these studies and because of some inconsistency in the results.
Recommendation
1.5. We suggest that appreciable resources be devoted to patient-mediated interventions (Grade 2B).
1.6. Audit and feedback
Seven RCTs406267686970 evaluated the effect of audit and feedback on guideline uptake. Analogous to the interventions described above, these RCTs predominantly took place in primary care settings and were not related to antithrombotic or thrombolytic therapy. For example, interventions focused on compliance with test-ordering guidelines or workup of low hemoglobin levels.69 Only one study40 reported concealment of allocation, three studies406870 reported complete follow-up, and one study67 reported blinded assessment of the outcomes. All RCTs reported improvement on process of care, and on average the effect was modest, but not all RCTs reported significant improvements. There were insufficient data for assessment of effects on patient outcomes. The survey indicated that audit and feedback are associated with limited feasibility, acceptability, and relatively high cost.
Recommendation
1.6. We suggest that few resources be devoted to audit and feedback (Grade 2B).
2.0 Conclusions and Summary
We evaluated studies from an existing systematic review on guideline implementation. Although strategies can be broadly categorized, the existing body of literature on guideline implementation is inconsistent with respect to study quality, heterogeneous with respect to the specific implementation strategies evaluated, and predominantly based in the primary care setting. The largest number of high quality RCTs is available for computer reminders, which demonstrate their efficacy for changing process of care and outcomes. High-quality studies for other implementation strategies, such as educational meetings, are limited. Therefore, consideration of evidence, values, and preferences for most interventions in this article generates Grade 2B recommendations.
There are few studies describing guideline implementation strategies in the field of antithrombotic or thrombolytic therapy, and we did not identify RCTs in this area fulfilling our inclusion criteria. While it is conceivable that one could generalize the results of studies that evaluate guideline implementation strategies for conditions unrelated to thrombosis, culture and practice patterns of different specialty disciplines may require different strategies. We also identified studies evaluating one of the six implementation strategies to promote evidence-based thrombosis care not tested in RCTs or not based on clinical guidelines (that would not have been included in the review by Grimshaw et al14) by searching the EPOC specialized register. Our search yielded only 23 studies. Although we reviewed an existing comprehensive database of studies, we may have missed recent studies. The modest size of the literature base is surprising, since the field of thrombosis research offers a plentitude of efficacious interventions with high-quality evidence, and therefore represents an ideal discipline for research on guideline implementation.
There is little original research on the feasibility, acceptability, and cost of implementation strategies in thrombosis research. Our survey of the ACCP conference attendants provides some insights. The most familiar and prevalent implementation strategies are dissemination of educational material and educational meetings. Respondents viewed these implementation strategies as the most feasible (score 5.8), acceptable (score 5.8), and least costly (score of 4.6) of all implementation strategies. However, they considered audit and feedback the least feasible (score of 3.5), least acceptable (score of 3.6), and most costly (score of 3.1) implementation strategy. Paradoxically, research suggests that audit and feedback may be an effective behavior change strategies.14 Implementation strategies that have the opportunity to modify prescribing behavior at the point of care, such as computer reminders, were considered to be feasible (score of 4.1) and highly acceptable to clinicians (score of 4.7); however, the success of this implementation strategy may be limited by costs, particularly in health-care institutions without clinical information systems (score of 3.0).
The implications of this literature review differ slightly for individual users of the ACCP antithrombotic and thrombolytic guidelines. Our findings provide support that individual clinicians and group practices can use simple implementation strategies such as computer reminders and patient-mediated interventions in their practice to improve the process of care. For experts and researchers in the field of thrombosis research, this literature summary underscores the need for additional high-quality studies in the area of guideline implementation and uptake. This research should focus on ascertaining whether those strategies for which high-quality evidence is currently limited are effective, and exploring the feasibility and effectiveness of multifaceted strategies. The ACCP guidelines on antithrombotic and thrombolytic therapy provide ample examples of strategies that could be tested. Conducting controlled trials using these ACCP guidelines, ideally based on interventions that lead to Grade 1A recommendations, could represent productive guideline evaluation research programs. For example, the Agency for Healthcare Research and Quality translating research into practice initiative funded 27 implementation studies in 1999/2000. The PPRNet-TRIP Project: Primary and Secondary Prevention of Coronary Heart Disease and Stroke is an RCT71 funded by this initiative to evaluate the impact of academic detailing (educational outreach) and electronic medical records on adherence with clinical practice guidelines across 23 primary care settings across the United States. Another model is the conduct of implementation research in tandem with the conduct of explanatory randomized trials in antithrombotic and thrombolytic therapy.
For administrators and clinical directors, this review provides support that guideline-implementation strategies can improve process of care. In many health-care settings, some of these strategies could have simple implementation protocols, such as computer reminders. Other strategies that could help clinicians improve VTE prophylaxis is participation in VTE registries. US and international thromboembolism registries offer insight into current anticoagulation practices, particularly with regard to preventive strategies. In the DVT-FREE Registry, 5,451 patients with ultrasound-proven deep venous thrombosis were enrolled at 183 sites in the United States over a 4-month period. Of the 2,726 patients who received a diagnosis as inpatients, only 1,147 patients (42%) received prophylaxis within the 30 days prior to the diagnosis.72 In the International Medical Prevention Registry on Venous Thromboembolism,73 only 38% of hospitalized medical patients enrolled received some form of prophylaxis. These registry data further emphasize the need for advances in guideline implementation.
What are the necessary steps for guideline implementation and implementation research? To improve patient outcomes and advance research on guideline implementation, a first step is the demonstration that a gap between practice and available evidence exists. One can categorize reasons for poor guideline uptake into groupings of evidence gaps, knowledge gaps, and clinician attitude. Evidence gaps, typically represented by grade B and grade C recommendations in the ACCP grading methodology, require additional explanatory trials. Attaining local "buy-in" after dissemination, preferably with the help of local opinion leaders, and achieving consensus on the guidelines themselves are important steps to overcome attitudes that prevent guideline uptake and ensure successful implementation. Using reminders or computer messages as well as audit and feedback of individual performance are additional steps that can increase guideline uptake and address gaps in the process of guideline implementation. Public awareness about thrombotic disease could further support guideline implementation. For example, the Center for Disease Control and Prevention and the American Public Health Association co-hosted the Public Health Leadership Conference on Deep-Vein Thrombosis, bringing together > 60 organizations to discuss the urgent need to make deep vein thrombosis a major US public health priority. One of the outcomes of the Leadership Conference was the development of a White Paper sponsored by the American Public Health Association. The White Paper, entitled "Deep-Vein Thrombosis: Advancing Awareness to Protect Patient Lives," is available on the world wide Web.74
| SUMMARY OF RECOMMENDATIONS |
|---|
|
|
|---|
1.2. We suggest that few resources be devoted to educational meetings (Grade 2B).
1.3. We suggest that few resources be devoted to educational outreach visits (Grade 2B).
1.4. We suggest that appreciable resources be devoted to computer reminders (Grade 2A).
1.5. We suggest that appreciable resources be devoted to patient-mediated interventions to encourage uptake of the guidelines (Grade 2B).
1.6. We suggest that few resources be devoted to audit and feedback (Grade 2B).
| Footnotes |
|---|
| References |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
V. F. Tapson, H. Decousus, M. Pini, B. H. Chong, J. B. Froehlich, M. Monreal, A. C. Spyropoulos, G. J. Merli, R. B. Zotz, J.-F. Bergmann, et al. Venous Thromboembolism Prophylaxis in Acutely Ill Hospitalized Medical Patients: Findings From the International Medical Prevention Registry on Venous Thromboembolism Chest, September 1, 2007; 132(3): 936 - 945. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. Grupper, A. Grupper, D. Rudin, B. Drenger, D. Varon, D. Gilon, Y. Gielchinsky, M. Menashe, Y. Mintz, A. Rivkind, et al. Prevention of perioperative venous thromboembolism and coronary events: differential responsiveness to an intervention program to improve guidelines adherence Int. J. Qual. Health Care, April 1, 2006; 18(2): 123 - 126. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME |