(Chest. 2004;126:174S-178S.)
© 2004
American College of Chest Physicians
Methodology for Guideline Development for the Seventh American College of Chest Physicians Conference on Antithrombotic and Thrombolytic Therapy
The Seventh ACCP Conference on Antithrombotic and Thrombolytic Therapy
Holger J. Schünemann, MD, MSc, PhD, FCCP;
Heather Munger, MLS;
Stewart Brower, MLIS;
Martin O’Donnell, MD;
Mark Crowther, MD, MSc;
Deborah Cook, MD, MSc and
Gordon Guyatt, MD, MSc, FCCP
Correspondence to: Holger Schünemann, MD, PhD, FCCP, Department of Medicine, Division of General Internal Medicine, University at Buffalo, ECMC-CC 142, 462 Grider St, Buffalo, NY 14226: e-mail: hjs{at}buffalo.edu
 |
Abstract
|
|---|
This article describes the methodology for the Seventh American College of Chest Physicians (ACCP) Conference on Antithrombotic and Thrombolytic Therapy: Evidence-Based Guidelines. Guideline authors began by specifying the population, the intervention and alternative, and the outcomes for each clinical question, and defined the criteria for eligible articles, including methodological criteria, for each recommendation. Librarians, in collaboration with guideline authors and methodologists, conducted systematic searches for evidence. Guideline authors systematically evaluated the evidence, considered the full range of benefits, risks, inconvenience, and costs associated with alternative management strategies, considered patients’ underlying values and preferences, and made recommendations accordingly. To increase the likelihood that the recommendations adequately represent patient values and preferences, the development process included a review of recommendations by research methodologists, practicing generalists, and specialists. Chapters are organized so that evidence is clearly linked to the relevant recommendations, and that recommendations particularly sensitive to underlying values and preferences are explicitly identified. Authors paid careful attention to the strength of the underlying evidence and to the balance between risks and benefits, which are both reflected in the grades of recommendations. Thus, improvements to the process of making recommendations for the ACCP guidelines include the explicit definition of questions, transparent eligibility criteria for including studies, and the specification of values and preferences underlying recommendations where they are particularly relevant. In combination with our previous practice of grading recommendations according to their strength, and the methodological quality of the supporting studies, these innovations establish our guidelines as, by and large, evidence based.
|
Introduction
|
|---|
The methodology for the Seventh American College of Chest Physicians (ACCP) Conference on Antithrombotic and Thrombolytic Therapy: Evidence-Based Guidelines differed from the previous conferences in several respects. As with other iterations, the changing evidence base in the field of antithrombotic and thrombolytic research led to many updated and new recommendations. However, users of these guidelines will easily recognize another change. The new title of the conference conveys the emphasis on an evidence-based approach to making recommendations that has been part of the antithrombotic guidelines from the beginning but has received increased emphasis with every subsequent iteration.
Evidence-based approaches include the acknowledgment of factors other than evidence that inevitably influence recommendations (ie, values and preferences) and that other factors that ideally would not, but might (ie, conflicts of interest). In this edition of the guidelines, we have developed explicit rules for conflicts that participants must declare, and we list these prominently in the front matter of the guideline document.1
The development of evidence-based guidelines includes explicitly defining the question that the guideline or recommendation is addressing, formulating eligibility criteria for the evidence to be considered, conducting a comprehensive search for evidence, evaluating study quality, summarizing the studies, balancing the benefits and downsides of the alternative management strategies, and, finally, acknowledging the values and preferences underlying the recommendations.2 This process ends with a recommendation for action, and a grading of that recommendation according to the balance of benefits and downsides, and the methodological quality of the evidence. In this article, we describe the methodology for guideline development for the Seventh ACCP Conference on Antithrombotic and Thrombolytic Therapy. Figure 1
summarizes this process.
View larger version (38K):
[in this window]
[in a new window]
[Download PPT slide]
|
Figure 1. Four editors (two methodologists and two content experts) coordinated the writing of chapters for the Seventh ACCP Conference on Antithrombotic and Thrombolytic Therapy: Evidence-Based Guidelines. After identification of search and eligibility criteria by chapter authors, librarians in collaboration with two of the editors searched for evidence. Trained literature screeners reviewed citations and removed irrelevant citations under supervision of one editor. Chapter authors updated previous chapters and completed the writing of the chapters in close collaboration with all editors and authors of other chapters.
|
|
|
1.0 Defining the Clinical Question
|
|---|
Developing a clinical practice guideline should begin with specifying a clinical question that defines the relevant population, alternative management strategies, and outcomes.3 For the current ACCP guidelines, authors defined one question for each recommendation or set of recommendations. Readers can find these questions in the corresponding table of each article containing practice recommendations.
|
2.0 Presentation of Evidence and Recommendations
|
|---|
To provide a transparent, explicit link among questions, evidence, and recommendations, the section numbering in each chapter corresponds to numbers in the corresponding table in the articles that specifies the patients, interventions, and outcomes. The section numbering also corresponds to the numbering of the recommendations themselves.
|
3.0 Process of Searching for Evidence
|
|---|
Defining the clinical question provided the framework for formulating eligibility criteria that guided the search for relevant evidence. Prior to searching for the evidence, methodological experts and librarians reviewed each question to ensure that the librarians could derive a comprehensive search strategy. For example, for questions about antiplatelet agents, the search included the relevant antiplatelet agents after discussion with the article authors. More specifically, authors then decided whether to include dipyridamole in a search that already included aspirin, clopidogrel, and ticlopidine.
In specifying eligibility criteria, authors not only identified patients, interventions, and outcomes, but also methodological criteria. For most therapeutic studies, authors restricted eligibility to randomized controlled trials (RCTs). For example, Albers et al4 considered whether clinicians should offer thrombolytic therapy in acute stroke. These authors defined patients as anyone presenting with acute thrombotic stroke (divided into presentation of < 3 h and > 3 h after the onset of symptoms). They defined the intervention as any thrombolytic regimen compared to no intervention or placebo. They defined the outcome as death or functional status based on assessment with a validated functional status instrument. The methodology was restricted to RCTs. This question yielded several recommendations, including whether patients with acute ischemic stroke presenting within 3 h of symptom onset should receive IV tissue plasminogen activator.
For many questions, RCTs did not provide sufficient data, and article authors also included observational studies. This was also true when randomized trials were not the most appropriate design to use for addressing the research question. In particular, randomized trials are not necessarily the best design to understand risk groups (eg, the baseline or expected risk of a given event for certain subpopulations). Because there are no interventions examined in questions about prognosis, one replaces interventions by the exposure, which is time. For example, to obtain information about the risk of ischemic stroke in patients with atrial fibrillation in specific risk groups, the sensible question was, in patients with atrial fibrillation differing in age, BP, left ventricular function, or history of previous embolic events, what is the risk of stroke or death over a given time period?
|
4.0 Identifying the Evidence
|
|---|
To identify the relevant evidence, a team of librarians at the University at Buffalo conducted comprehensive literature searches. For each question the authors provided, the librarians developed sensitive (but not specific) search strategies, including all languages, and conducted separate searches for systematic reviews, RCTs, and, if applicable, observational studies. The librarians searched the Cochrane Database of Systematic Reviews, the Database of Abstracts of Reviews of Effectiveness and Cochrane Register of Controlled Trial, the ACP Journal Club, MEDLINE, and Embase for studies published between 1966 and June 2002 in any language. To filter MEDLINE and Embase search results for RCT evidence, the librarians used the search strategy developed by the Cochrane Collaboration (full strategy available in Appendix online at: http://www.chestjournal.org/content/vol126/3_suppl_1).
For observational studies, they restricted their searches to human studies. Searches were not further restricted in terms of methodology. While increasing the probability of identifying all published studies, this sensitive approach resulted in large number of citations for many of the defined clinical questions. Therefore, trained research assistants screened the citation list developed from the search and removed any apparently irrelevant citations. These irrelevant citations included press news, editorials, narrative reviews, single case reports, animal studies (any nonhuman studies), and letters to the editor. Authors included data from abstracts of recent meetings if reporting was transparent and all necessary data for the formulation of a recommendation were available. We did not explicitly use Internet sources to search for research data.
|
5.0 Standard Consideration of Study Quality
|
|---|
High-quality clinical guidelines pay careful attention to the methodological quality of the studies that form the basis of their recommendations. Another article in this Supplement describes in detail the basic grading of methodological quality.5 In brief, consistent results from RCTs result in grade A recommendations, and observational studies with very strong effects or secure generalization from RCTs result in grade C+. Inconsistent results from RCTs yield grade B, and observational studies result in grade C.
For some recommendations, authors were interested in defining the baseline risk of a specific population. To determine the baseline risk for recommendations, only studies that met specific criteria were eligible. Generally, these included the following: (1) cohort studies or control groups of RCTs; (2) reporting of at least 200 participants (cohort studies) or 200 patients in the control group of RCTs; (3) focus on populations that were similar to the population for which the guideline authors were making recommendations; (4) sufficient length of follow-up to cover the natural course of the disease; and (5) a < 20% loss of patients to follow-up.
|
6.0 Summarizing Evidence
|
|---|
The electronic searches also included searching for systematic reviews. If authors were satisfied with a recent high-quality systematic review, evidence from that review provided a foundation for the relevant recommendation. For example, Albers and colleagues4 utilized a systematic review and meta-analysis as the foundation for their recommendation on IV streptokinase therapy for acute ischemic stroke between 0 and 6 h after symptom onset (see article in this supplement on stroke [section 1.3]). Geerts et al6 utilized several meta-analyses for their recommendations (see article in this supplement on the prevention of venous thromboembolism [section 2]).
Pooled analyses from high-quality systematic reviews formed, wherever possible, the evidence base of the recommendations. Pooling offers the advantage of obtaining more precise estimates of treatment effects and allows for a greater generalizability of results. However, pooling also bears the risk of spurious generalization. In general, the summary estimates of interest were the different types of outcomes conveying benefit and downsides (ie, risk, burden, and cost).
|
7.0 Acknowledge Values and Preferences Underlying Recommendations
|
|---|
Under ideal circumstances, patients’ values and preferences would be available for every recommendation and then integrated into the recommendations that guideline developers make. Were this the case, guideline developers could describe in detail the underlying value judgments on which they are based. In practice, values and preferences are rarely available. Therefore, many guideline reports and their users assume that guideline developers adequately represent patients’ interests when they make a health-care recommendation.7
To increase the likelihood that the recommendations adequately represent patient values and preferences, the development process included a review of recommendations by research methodologists, practicing generalists, and specialists. Authors and editors tried to identify recommendations that were particularly sensitive to values and preferences, and to specify these underlying values and preferences. For example, Singer et al8 recommended the use of aspirin over warfarin for the prevention of stroke in patients with chronic or intermittent atrial fibrillation who are < 65 years of age and have no additional risk factors. They noted that "individual lower-risk patients may rationally choose anticoagulation to gain greater protection against ischemic stroke if they value protection against stroke much higher than reducing risk of hemorrhage and the burden of managing anticoagulation."
|
8.0 Grade Strength of Recommendation
|
|---|
A systematic approach to grading the strength of treatment recommendations can minimize bias and aid the interpretation of treatment recommendations. The authors of articles in this Supplement have graded their recommendations as strong (ie, grade 1, benefits much greater than risks or risks much greater than benefits), wording the recommendation accordingly as "We recommend," or weak (ie, grade 2, benefits not clearly greater or less great than risks), wording the recommendation as "We suggest." They also have graded the methodological quality of the underlying evidence. Another article in this supplement presents in detail our approach to grading recommendations.9
|
9.0 Finalizing and Harmonizing Recommendations
|
|---|
After completing the steps we have described above, the guideline authors formulated draft recommendations prior to the conference that served as the foundation for authors to work together and critique the recommendations. Drafts of all articles including draft recommendations were available for review during the conference. A representative of each article presented potentially controversial issues in their recommendations at plenary meetings. Article authors met to integrate feedback, to consider related recommendations in other articles, and to revise their own guidelines accordingly. Authors continued this process after the conference until they reached agreement within their groups and with other author groups who had provided critical feedback. Finally, the editors of this supplement harmonized the articles and resolved the remaining disagreements between articles through facilitated discussion.
|
10.0 Limitations of These Guideline Development Methods
|
|---|
The limitations of these guidelines include the possibility that some authors followed this methodology more closely than others, although the development process was centralized and supervised by the editors. Second, it is possible that we missed relevant studies despite the comprehensive searching process. Third, we did not centralize the methodological evaluation of all studies to facilitate uniformity in the validity assessments of the research incorporated into these guidelines. Fourth, if high-quality meta-analyses were unavailable, we did not statistically pool primary study results using meta-analysis. Finally, sparse data on patient preferences and values, resources, and other costs represent additional limitations that are inherent to most guideline development methods.
|
11.0 Future Directions of ACCP Guidelines
|
|---|
Future iterations of the current guidelines will tackle the limitations mentioned above. For example, we asked authors making clinical recommendations to consider concealment, blinding, loss to follow-up, and consistency of the results when assigning grade A or grade B to a given recommendation. However, as described above, the evaluation of these methodological criteria was largely at the discretion of each of the authors of an article. As such, a recommendation of 1A or 1B may represent some variation in treatment effect and quality of the studies supporting that recommendation. To further improve the quality of these evidence-based recommendations, our next objective is to perform additional evaluations, preferably supervised and coordinated centrally, of the quality of all trials cited in this document (assessment criteria available online at http://www.chestjournal.org/content/vol126/3_suppl_1). This initiative will enhance the consistency and transparency of the approach to assigning grade A and grade B recommendations for the Eighth ACCP Conference on Antithrombotic and Thrombolytic Therapy Guidelines.
|
12.0 Conclusion
|
|---|
Improvements to the process of making recommendations for the ACCP guidelines have included the explicit definition of questions, transparent eligibility criteria for including studies, the methodological evaluation of the RCTs included, and the specification of the values and preferences underlying recommendations where particularly relevant. In combination with our previous practice of grading recommendations according to their strength, and the methodological quality of the supporting studies, these innovations establish our guidelines as, by and large, evidence-based. In addition, the organization of the guidelines allows clinicians to quickly establish the link between evidence and recommendations. Future work will include a detailed assessment of the specific methodological criteria of individual randomized trials underlying grade A or B recommendations and enhancements in integrating patients’ preferences and values.
|
Footnotes
|
|---|
Abbreviations: ACCP = American College of Chest Physicians; RCT = randomized controlled trial
|
References
|
|---|
- Hirsh, J, Guyatt, G, Albers, GW, et al (2004) Seventh ACCP Conference on Antithrombotic and Thrombolytic Therapy: Evidence-Based Guidelines. Chest 126(suppl),172S-173S
- Guyatt, GRH, Richardson, W, Green, L, et al Moving from evidence to action. Guyatt, GRD eds. Users’ guide to the medical literature: a manual for evidence-based clinical practice 2002,175-204 AMA Press. Chicago, IL:
- McKibbon, A, Hunt, D, Richardson, SW, et al Finding the evidence. Guyatt, GRD eds. Users’ guides to the medical literature: a manual for evidence-based clinical practice 2002,16 American Medical Association Press. Chicago, IL:
- Albers, GW, Amarenco, P, Easton, JD, et al Antithrombotic and thrombolytic therapy for ischemic stroke. Chest 2004;126(suppl),483S-512S
- Guyatt, GRH, Schünemann, D, Cook, R, et al Grades of recommendation for antithrombotic agents. Chest 2004;126(suppl),483S-512S
- Geerts, W, Pineo, G, Heit, J, et al Prevention of venous thromboembolism. Chest 2004;126(suppl),338S-400S
- Shekelle, PG, Woolf, SH, Eccles, M, et al Clinical guidelines: developing guidelines. BMJ 1999;318,593-596[Free Full Text]
- Singer, DE, Albers, GW, Dalen, JE, et al Antithrombotic therapy in atrial fibrillation. Chest 2004;126(suppl),429S-456S
- Guyatt, G, Schünemann, H, Cook, D, et al Applying the grades of recommendation for antithrombotic and thrombolytic therapy. Chest 2004;126(suppl),179S-187S
This article has been cited by other articles:
|
|
|
|
 
A. Miller, I. Campbell, T. Fennerty, G. E. Raskob, and H. R. Buller
Duration of warfarin in pulmonary embolism.
Chest,
July 1, 2006;
130(1):
299 - 300.
[Full Text]
[PDF]
|
|
|
|
|
|
|
D. W Kubiak, P. M Szumita, and J. R Fanikos
Ann. Pharmacother.,
November 1, 2005;
39(11):
1956 - 1957.
[Full Text]
[PDF]
|
|
|
TOP
Abstract
Introduction
