(Chest. 2005;127:1627-1630.)
© 2005
American College of Chest Physicians
Clinical Gestalt and the Diagnosis of Pulmonary Embolism*
Does Experience Matter?
Christopher Kabrhel, MD;
Carlos A. Camargo, Jr, MD, DrPH, FCCP and
Samuel Z. Goldhaber, MD, FCCP
* From the Department of Emergency Medicine (Drs. Kabrhel and Camargo), Massachusetts General Hospital; and Cardiovascular Division (Dr. Goldhaber), Department of Medicine, Brigham & Women’s Hospital, Harvard Medical School, Boston, MA.
Correspondence to: Christopher Kabrhel, MD, Department of Emergency Medicine, Massachusetts General Hospital, 55 Fruit St, Clinics Building 115, Boston, MA 02114; e-mail: ckabrhel{at}partners.org
 |
Abstract
|
|---|
Study objectives: We sought to determine whether the accuracy of pretest assessment of the likelihood of pulmonary embolism (PE) was related to physician experience. We compared the accuracy of the subjective pretest probability assessment made by senior physicians (postgraduate year [PGY]-4+) to that of interns (PGY-1) and residents (PGY-2 and PGY-3) working in the emergency department of a large teaching hospital.
Design: Prospective observational study.
Setting: Urban, academic emergency department with an annual census of 48,000 patient visits.
Patients: Five hundred eighty-three adults evaluated for PE in the emergency department.
Interventions: Eligible patients had at least one diagnostic test ordered to workup PE. The physician treating the patient was asked whether he or she considered PE the most-likely diagnosis or whether an alternative diagnosis was most likely. This result was compared to the ultimate diagnosis. Physician experience was categorized by the number of years of training since medical school graduation.
Measurements and results: There was a trend toward increasing accuracy with increasing experience, demonstrated by the frequency of true-positive assessments (17% vs 20% vs 25%), true-negative assessments (89% vs 94% vs 96%), and likelihood ratio (1.49 vs 2.34 vs 3.33), respectively.
Conclusions: Accurate determination of the pretest probability of PE appears to increase with clinical experience. However, the difference in accuracy between inexperienced and experienced physicians is not sufficiently large to distinguish between the two when determining whether clinical gestalt or a clinical prediction rule should be used to determine the pretest probability of PE.
Key Words: clinical gestalt • clinical prediction rule • education • pretest probability assessment • pulmonary embolism
|
Introduction
|
|---|
Pulmonary embolism (PE) continues to pose a diagnostic challenge for physicians. The high mortality rate and nonspecific symptoms of PE lead us into the undesirable position of simultaneously overtesting12 and underdiagnosing34 the disease. Attempting to remedy this situation, several authors have developed clinical prediction rules to categorize patients as low, intermediate, or high pretest probability for PE.5678 These rules are intended to guide diagnostic testing. Several studies910111213141516 have shown that clinical prediction rules perform no better than subjective clinical gestalt. They are, however, perceived to have the advantage of being independent of physician experience, whereas gestalt is believed to vary with experience. We sought to determine whether the accuracy of clinical gestalt improved with physician experience. To do this, we compared the accuracy of the subjective pretest probability assessment of PE likelihood made by attending physicians to that of interns and residents working in the emergency department of a large teaching hospital.
|
Materials and Methods
|
|---|
The Human Research Committee of Partners HealthCare System approved this study. We prospectively enrolled 583 patients who presented to an urban teaching hospital emergency department and were evaluated for possible PE. To be eligible, patients had at least one diagnostic test ordered to workup PE. Baseline characteristics of enrolled patients were retrieved from the computerized patient record. Tests included a rapid enzyme-linked immunosorbent assay (ELISA) d-dimer (VIDAS; bio Merieux SA; Marcy-Etoile, France), multidetector contrast-enhanced PE protocol CT (SOMATOM Sensation 4; Siemens; Erlangen, Germany), ventilation/perfusion scan (E-cam dual head; Seimens), or pulmonary angiography. Patients were considered positive for PE if they had positive pulmonary angiogram, high-probability ventilation/perfusion scan, or positive PE protocol CT scan findings. Patients were considered to have PE excluded if they had negative pulmonary angiogram, negative ELISA d-dimer (defined as < 500 ng/mL), negative PE protocol CT scan, or normal or low-probability ventilation/perfusion lung scan findings The use of a negative rapid ELISA d-dimer as a single test to exclude PE has been described elsewhere.17181920 This test has also been studied in our institution and was found to rule out PE with high sensitivity (96.4%) and negative predictive value (99.6%).19 Patients with an intermediate-probability ventilation/perfusion lung scan result who did not undergo CT or pulmonary angiography were considered negative for PE if they had a negative lower-extremity venous ultrasound finding, and were considered positive for PE if they had a positive lower-extremity venous ultrasound finding. Patients were followed up for 3 months. After ordering a test to evaluate PE, but before receiving results, the physician enrolling the patient was asked whether he or she considered PE the most-likely diagnosis or whether an alternative diagnosis was most likely. For alternative diagnoses, clinicians were provided a list of diagnoses: angina or acute coronary syndrome; anxiety or panic attack; asthma or COPD; congestive heart failure or pulmonary edema; musculoskeletal pain; pleural effusion; pneumonia; pneumothorax; viral syndrome; or upper respiratory infection. This result was compared to the patient’s ultimate diagnosis as determined by objective testing using the PE evaluation algorithm described above as the criterion standard.
We analyzed the accuracy of clinical gestalt for three groups of physicians: physicians training in either emergency medicine, general surgery, or internal medicine who were in their first year of practice since graduating medical school (postgraduate year [PGY]-1); physicians training in emergency medicine, general surgery, or internal medicine who were in their second or third year of practice since graduating medical school (PGY-2 and PGY-3); and emergency physicians who completed at least 3 years of emergency medicine training since graduating medical school (PGY-4+). A true-positive (TP) case was defined as a patient for whom the clinician thought PE was the most likely diagnosis, who had PE diagnosed according to the methodology above. A true-negative (TN) case was defined as a patient for whom the clinician thought PE was not the most likely diagnosis, who had PE excluded according to the methodology above.
Statistical analysis was performed using software (Intercooled Stata version 8.0; StataCorp; College Station, TX). The association between PE diagnosis and other factors was examined using 
2 test or student t test as appropriate. All p values are two-tailed, with p < 0.05 considered statistically significant.
|
Results
|
|---|
We enrolled 583 patients (Table 1
) over a period of 13 months. Of these, 89% underwent d-dimer testing. The d-dimer results were negative in 45% of those tested. No patient with a negative d-dimer result had PE on 3-month follow-up. Of the 583 patients enrolled, PE was diagnosed in 59 patients (10%). PE was considered the most likely diagnosis for 153 patients (26%). Of these, PE was actually diagnosed in 32 patients (21%). Of the 430 patients for whom PE was not considered the most likely diagnosis, PE was diagnosed in 27 patients (6.3%).
Across experience level, there were no differences in the percentage of patients for whom physicians considered PE most likely (Table 2 ). However, the frequency of PE in patients for whom the physician thought PE was most likely was about twice that of the overall population. The TP rates across experience level were quite low (17 to 25%), while we noted that the TN rates across experience levels were quite high (89 to 96%). There was a trend toward increasing accuracy with experience (p for trend, 0.10). These measures are greatly dependent on the frequency of disease, which was 10%. We therefore chose to compare likelihood ratios, which are also needed to calculate the Bayesian posttest probability of PE.
The likelihood ratio for physicians with 
4 years of experience was more than twice that of interns (3.33 vs 1.49). Analysis performed using alternative groupings, including months since medical school graduation, yielded similar results (data not shown). Additionally, results of a post hoc analysis, which included patients with positive lower-extremity ultrasound result but no chest imaging as having PE, did not differ from results presented in Table 2 (data not shown).
|
Discussion
|
|---|
The use of clinical gestalt in determining the pretest probability of PE has received much attention in the medical literature since the publication of the Prospective Investigation of Pulmonary Embolism Diagnosis in 1990.45679101415212223 Clinical gestalt appears to be quite variable.9 Positive likelihood ratios range from 1.4 to 29.9 The ability of clinicians to apply their clinical gestalt appears to depend on several factors. Clinician experience is thought to play a large role, although this has never been studied prospectively. In order to address this variability across experience level, and to make pretest probability assessment more objective, much recent attention has been focused on developing clinical prediction rules. However, while commonly used clinical prediction rules have been shown to be accurate, they do not yet appear to perform better than subjective clinical gestalt.9 However, they are perceived to have the advantage of being independent of the experience of the physician.
We found that the subjective clinical gestalt of experienced physicians may be more accurate at correctly categorizing patients evaluated for PE than the clinical gestalt of inexperienced physicians. The likelihood ratio for experienced physicians was more than twice that of interns. Sensitivity, specificity, positive predictive value, negative predictive value, the percentage of TP results, and the percentage of TN results also appeared to improve with experience, although confidence intervals overlapped.
To determine the clinical significance of these findings, we compared the likelihood ratios of interns and experienced physicians in our study to likelihood ratios reported for prospectively validated clinical prediction rules. The purpose was to determine whether the results of any of these studies suggest a cutoff whereby inexperienced physicians should use a clinical score while experienced physicians may use their clinical gestalt. Six such prospective validation studies were identified. Two studies915 report likelihood ratios (1.4, 1.4) that are lower than that of the interns in our study. Four studies212223 report likelihood ratios (5.8, 5.9, 5.9, and 29) that are higher than that of the experienced physicians in our study. No study reports a likelihood ratio between those of the interns and the senior physicians in our study. Our data suggest a trend toward increasing accuracy of clinical gestalt with experience. However, this trend is not sufficiently large to warrant drawing a distinction between inexperienced and experienced physicians in terms of who should use a clinical prediction rule.
Our study is limited by the relatively small number of patients in each group. We used "PE most likely" as a surrogate for high pretest probability. However, we offered clinicians the opportunity to select from a list of alternative diagnoses that included several subjective diagnoses that are not life threatening (eg, musculoskeletal pain). Therefore, we believe clinicians who answered that PE was most likely did not choose PE because it was the most likely "dangerous" diagnosis, but because it was in fact the most likely diagnosis in their opinion. We also used somewhat arbitrary definitions of "experienced" and "inexperienced" physicians. We defined experienced physicians as physicians with at least 3 full years of clinical training since graduating from medical school. This cutoff was chosen as it is the duration of training required to be eligible to be board certified by both the American Board of Emergency Medicine and the American Board of Internal Medicine.
|
Conclusions
|
|---|
The ability to accurately determine the pretest probability of PE appears to increase with clinical experience. However, the difference in accuracy between inexperienced and experienced physicians is not sufficiently large to distinguish between the two when determining whether clinical gestalt or a clinical prediction rule should be used to determine the pretest probability of PE.
|
Footnotes
|
|---|
Abbreviations: ELISA = enzyme-linked immunosorbent assay; PE = pulmonary embolism; PGY = postgraduate year; TN = true-negative; TP = true-positive
Dr. Camargo is supported by National Institutes of Health grant HL-63841.
Received for publication April 27, 2004.
Accepted for publication November 9, 2004.
|
References
|
|---|
- Trowbridge, RL, Araoz, PA, Gotway, MB, et al (2004) The effect of helical computed tomography on diagnostic and treatment strategies in patients with suspected pulmonary embolism. Am J Med 116,84-90[CrossRef][ISI][Medline]
- Hainaut, P, Elamly, A, Dessomme, B, et al ELISA d-dimer measurement for the clinical suspicion of pulmonary embolism in the emergency department: one-year observational study of the safety profile and physician’s prescription. Acta Clin Belg 2003;58,233-240[ISI][Medline]
- Lindblad, B, Sternby, NH, Bergqvist, D Incidence of venous thromboembolism verified by necropsy over 30 years. BMJ 1991;302,709-711[ISI][Medline]
- Ryu, JH, Olson, EJ, Pellikka, PA Clinical recognition of pulmonary embolism: problem of unrecognized and asymptomatic cases. Mayo Clin Proc 1998;73,873-879[ISI][Medline]
- Wells, PS, Anderson, DR, Rodger, M, et al Use of a clinical model for safe management of patients with suspected pulmonary embolism. Ann Intern Med 1998;129,997-1005[Abstract/Free Full Text]
- Wells, PS, Anderson, DR, Rodger, M, et al Derivation of a simple clinical model to categorize patient’s probability of pulmonary embolism: increasing the model’s utility with the SimpliRED d-dimer. Thromb Haemost 2000;83,416-420[ISI][Medline]
- Wicki, J, Perneger, T, Junod, A, et al Assessing clinical probability of pulmonary embolism in the emergency ward: a simple score. Arch Intern Med 2001;161,92-97[Abstract/Free Full Text]
- Kline, JA, Nelson, RD, Jackson, RE, et al Criteria for the safe use of d-dimer testing in emergency department patients with suspected pulmonary embolism: a multicenter US study. Ann Emerg Med 2002;39,144-152[CrossRef][ISI][Medline]
- Chunilal, SD, Eikelboom, JW, Attia, J, et al Does this patient have pulmonary embolism? JAMA 2003;290,2849-2858[Abstract/Free Full Text]
- PIOPED Investigators. Value of the ventilation/perfusion scan in acute pulmonary embolism: results of the Prospective Investigation of Pulmonary Embolism Diagnosis (PIOPED). JAMA 1990;263,2753-2759[Abstract]
- Miniati, M, Pistolesi, M, Marini, C, et al Value of perfusion lung scan in the diagnosis of pulmonary embolism: results of the Prospective Investigative Study of Acute Pulmonary Embolism Diagnosis (PISA-PED). Am J Respir Crit Care Med 1996;154,1387-1393[Abstract]
- Perrier, A, Desmarais, S, Goehring, C, et al D-dimer testing for suspected pulmonary embolism in outpatients. Am J Respir Crit Care Med 1997;156,492-496[Abstract/Free Full Text]
- Perrier, A, Desmarais, S, Miron, MJ, et al Non-invasive diagnosis of venous thromboembolism in outpatients. Lancet 1999;353,190-195[CrossRef][ISI][Medline]
- Perrier, A, Bounameaux, H, Morabia, A, et al Diagnosis of pulmonary embolism by a decision analysis-based strategy including clinical probability, d-dimer levels, and ultrasonography: a management study. Arch Intern Med 1996;156,531-536[Abstract]
- Sanson, BJ, Lijmer, JG, Mac Gillavry, MR, et al Comparison of a clinical probability estimate and two clinical models in patients with suspected pulmonary embolism: ANTELOPE Study Group. Thromb Haemost 2000;83,199-203[ISI][Medline]
- Musset, D, Parent, F, Meyer, G, et al Diagnostic strategies for patients with suspected pulmonary embolism: a prospective multicentre outcome study. Lancet 2002;360,1914-1920[CrossRef][ISI][Medline]
- Perrier, A, Roy, PM, Aujesky, D, et al Diagnosing pulmonary embolism in outpatients with clinical assessment, d-dimer measurement, venous ultrasound, and helical computed tomography: a multicenter management study. Am J Med 2004;116,291-299[CrossRef][ISI][Medline]
- Perrier, A, Desmarais, S, Miron, MJ, et al Non-invasive diagnosis of venous thromboembolism in outpatients. Lancet 1999;353,190-195[CrossRef][ISI][Medline]
- Dunn, KL, Wolf, BS, Dorfman, DM, et al Normal d-dimer levels in emergency department patients suspected of acute pulmonary embolism. J Am Coll Cardiol 2002;40,1475-1478[Abstract/Free Full Text]
- Stein, PD, Hull, RD, Patel, KC, et al D-dimer for the exclusion of acute venous thrombosis and pulmonary embolism: a systematic review. Ann Intern Med 2004;140,589-602[Abstract/Free Full Text]
- Wells, PS, Anderson, D, Rodger, M, et al Excluding pulmonary embolism at the bedside without diagnostic imaging: management of patients with suspected pulmonary embolism presenting to the emergency department by using a simple clinical model and d-dimer. Ann Intern Med 2001;135,98-107[Abstract/Free Full Text]
- Kruip, MJHA, Slob, MJ, Schijen, JHEM, et al Use of a clinical decision rule in combination with d-dimer concentration in diagnostic workup of patients with suspected pulmonary embolism: a prospective management study. Arch Intern Med 2002;162,1613-1635
- Chagnon, I, Bounameaux, H, Aujesky, D, et al Comparison of two clinical prediction rules and implicit assessment among patients with suspected pulmonary embolism. Am J Med 2002;113,269-275[CrossRef][ISI][Medline]
TOP
Abstract
Introduction
