|
|
||||||||
Guest Access | Sign In via User Name/Password |
|||||||||
* From the University of Aberdeen (Dr. Price), Aberdeen, Scotland; National Jewish Medical and Research Center (Dr. Tinkelman), Denver, CO; and Cerner Health Insights (Drs. Nordyke, Isonaka, and Halbert), Beverly Hills, CA.
A list of COPD Questionnaire Study Group members is presented in the Appendix.
Correspondence to: R. J. Halbert, MD, MPH, Cerner Health Insights, 9100 Wilshire Blvd, Suite 655E, Beverly Hills, CA 90212; e-mail: rhalbert{at}cerner.com
Abstract
Objectives: In most primary care settings, spirometric screening of all patients at risk is not practical. In prior work, we developed questionnaires to help identify COPD in two risk groups: (1) persons with a positive smoking history but no history of obstructive lung disease (case finding), and (2) patients with prior evidence of obstructive lung disease (differential diagnosis). For these questionnaires, we now present a scoring system for use in primary care.
Methods: Scores for individual questions were based on the regression coefficients from logistic regression models using a spirometry-based diagnosis of obstruction as the reference outcome. Receiver operator characteristic analysis was used to determine performance characteristics for each questionnaire. Several simplified scoring systems were developed and tested.
Results: For both scenarios, we created a scoring system with two cut points intended to place subjects within one of three zones: persons with a high likelihood of having obstruction (high predictive value of a positive test result); persons with a low likelihood of obstruction (high predictive value of a negative test result); and an intermediate zone. Using these scoring systems, we achieved sensitivities of 54 to 82%, specificities of 58 to 88%, positive predictive values of 30 to 78%, and negative predictive values of 71 to 93%.
Conclusions: These questionnaires can be used to help identify persons likely to have COPD among specific risk groups. The use of a simplified scoring system makes these tools beneficial in the primary care setting. Used in conjunction with spirometry, these tools can help improve the efficiency and accuracy of COPD diagnosis in primary care.
Key Words: diagnostic techniques obstructive lung diseases primary care questionnaires sensitivity and specificity
Underdiagnosis of COPD is a widespread problem.1 Diagnostic confusion between asthma and COPD, while not as widespread, appears to be an important clinical problem in some patients.2 The definitive diagnostic maneuver for COPD is spirometry1; however, despite frequent advocacy for spirometric screening,34 spirometry is underused.5 This is especially true in the primary care setting, the usual site of initial presentation.56 There is a perception that spirometric screening of all at-risk persons is impractical in primary care. This has led to efforts to identify a subset of patients for whom such screening is likely to be cost-effective.7
In contrast to general population screening programs,8 efforts to locate persons within a primary care practice are more accurately referred to as case-finding programs.7 These types of programs should be directed toward groups known to have an increased prevalence of the condition to be identified. COPD prevalence is known to be increased in adults > 40 years old and in persons exposed to noxious smoke or fumes, especially cigarette smoke.1 Persons with prior evidence of respiratory problems but in whom a diagnosis has not been definitively established represent another group likely to benefit from closer scrutiny.
Previous work9 has shown that relatively simple questionnaires can help identify persons with an increased likelihood of fixed obstruction. Most recently, this has been demonstrated in two risk groups: (1) current and former smokers
40 years old with no prior evidence of obstructive lung disease (case-finding scenario)10; and (2) persons
40 years old with prior evidence of obstructive lung disease (differential diagnosis scenario).11 We now describe the development of a scoring system for these questionnaires suitable for use in a primary care setting.
Materials and Methods
Details of the development of the questionnaires have been described elsewhere.1011 In brief, two study sites (Aberdeen, Scotland and Denver, CO) were chosen for the evaluation. Subjects
40 years old were randomly selected from primary care practice rosters in these sites and invited by mail to participate in the study. Eligible respondents were enrolled after providing informed consent. Respondents were eligible if they reported the following: (1) a positive smoking history (current or former smokers), with no prior evidence of respiratory diagnosis (eg, no prior respiratory diagnosis and no respiratory medications within the past year); or (2) prior evidence of respiratory diagnosis (eg, any prior respiratory diagnosis or any respiratory medications within the past year), regardless of smoking status. Participants completed a questionnaire covering demographics and symptoms and then underwent spirometry with reversibility testing. Study diagnoses were based on guidelines developed by the Global Initiative for Chronic Obstructive Lung Disease1 and the Global Initiative for Asthma.12 A study diagnosis of COPD was assigned to persons with postbronchodilator FEV1/FVC ratio < 0.70. For the differential diagnosis analysis, a diagnosis of asthma was assigned to persons with postbronchodilator FEV1/FVC ratio
0.70 and FEV1 reversibility
200mL and
12% of baseline. Persons with no reversibility received a diagnosis of "probable asthma" if they had a prior diagnosis of asthma or were receiving long-term corticosteroids. The study was approved by ethics committees at the two sites. A description of the study populations is provided in Table 1
.
|
|
Results
The ROC curves are presented for the case-finding (Fig 1 ) and differential diagnosis (Fig 2 ) questionnaires, along with selected performance characteristics (Table 3 ). For each ROC curve, two cut points were determined by assessing the performance characteristics of the questions to choose those points representing the optimal combination of PPV, NPV, and the distribution of subjects between cut points. This system places subjects within one of three zones, depending on their likelihood of having obstruction: (1) increased likelihood of obstruction (optimum PPV); (2) intermediate likelihood of obstruction; or (3) decreased likelihood of obstruction (optimum NPV). Expressed another way, these zones represent, respectively, a probability of COPD that is (1) increased, (2) unchanged, or (3) reduced, compared to the risk group as a whole. Since PPV and NPV vary by prevalence, the scoring process was in part dependent on the baseline prevalence of obstruction within each group: 18.7% in the case-finding sample and 43.3% in the differential diagnosis sample.
|
|
|
|
|
Discussion
Prior work1011 has demonstrated that questionnaires based on patient-reported information can be used to identify persons likely to have COPD among specific risk groups. This study adds a simple scoring system to make these instruments practical for use in the primary care setting. Using this scoring system, these instruments demonstrated ROC performance comparable to that of other respiratory screening questionnaires, as illustrated in Table 6 . In COPD, for example, retrospective analyses using data from the Third National Health and Nutrition Examination Survey89 achieved sensitivities ranging from 54 to 86% and specificities from 40 to 71%. Symptom-based screening tools for asthma have been reported to achieve sensitivities from 38 to 80% in adults and 23 to 86% in children, with specificities ranging from 64 to 99% in adults and 55 to 100% in children.141516 These values compare well with airway hyperresponsiveness, which showed a sensitivity of 39%, specificity of 90%, PPV of 62%, and NPV of 78% when compared with physician-diagnosed asthma.17 Similar results have been described with recommended screening examinations for diseases such as breast18 and colorectal cancer.19
|
For policy purposes, we anticipated that the questionnaires would be used in diverse health-care contexts. We therefore tried to make the system adaptable to different regional needs. Taking into account the performance characteristics of the instruments, the three-zone approach was chosen to allow for some degree of flexibility in policy recommendation between different health systems, with different budgetary constraints and different policy priorities. These decisions (and subsequent policy recommendations) will also be affected by the clinical perception of the background prevalence of COPD in the group to whom the questionnaires are administered (ie, the prior probability of fixed obstruction).
We anticipate that persons assigned to the high-likelihood zones will be recommended for spirometry in most regions, and that persons in the low-likelihood zones will be deferred in most regions. Recommendations for the intermediate zones will depend on local conditions. In some regions, for example, persons falling within the intermediate zones will be recommended for spirometry with the intention of minimizing the number of cases missed. In regions with fewer health resources (or different health-spending priorities), persons assigned to this zone might be followed up clinically with spirometry deferred until a later date, so as to minimize the number of unnecessary procedures. By incorporating the scoring system described here, the questionnaires become powerful tools for helping the primary care clinician efficiently identify persons with obstructive lung disease.
Most physicians will note that they are likely to ask at least some of the questions included in our instruments as part of their routine workup. Placing these questions within the framework described here provides added value in at least two ways. First, because of complex interactions between question items, combining the questions into a single instrument changes their individual significance. The process described here helps identify the relationships between several key questions, providing an added value above the diverse questions asked by clinicians. Second, creating questionnaires with defined scoring and performance characteristics provides quantitative estimates of the risk of obstruction for the clinician.
Case-Finding Application
A key goal in seeking out previously undiagnosed cases of a disease among persons at known increased risk is to identify these new cases at the lowest incremental cost. The case-finding questionnaire could be very useful to enhance the efficiency of such current screening efforts using spirometry alone. We see at least two potential applications: (1) improving the efficiency of COPD diagnosis in smokers who present with respiratory symptoms (but no prior diagnosis), and (2) assisting in the early detection of COPD in smokers outside clinical settings.
In the first application, clinicians who feel that spirometric evaluation of all current and former smokers in their practice is impractical may use the questionnaire to "prescreen" those smokers at highest likelihood. This allows more efficient use of office and referral resources, minimizing the administrative burden on the practice. Since the questionnaire may be given unsupervised to patients in the waiting area, the only time required of the office staff is that required to calculate and interpret the results. The simplified scoring system presented here reduces this administrative burden considerably: a computerized version of the questionnaires could reduce it even further.
In the second application, the questionnaire could be used as part of an outreach program to screen smokers for potential lung health problems in settings outside the clinic, such as a worksite or general population screening program. These types of programs, often associated with a media awareness campaign or similar marketing outreach, can be very helpful in stimulating persons to consider the potential for lung damage due to smoking.20
In both these applications, the prevalence of COPD in the intermediate zone is expected to approximate 20%. Thus, the administrator of the program (a clinician in many cases) should determine in advance whether this prevalence is high enough to warrant follow-on evaluation with spirometry, and implement accordingly.
Differential Diagnosis Application
The differential diagnosis questionnaire could be useful as part of the evaluation process whenever there is lack of diagnostic clarity among patients suspected of having obstructive lung disease. This might include patients who have previously received respiratory medications without assignment of a specific diagnosis, or patients in whom the previously assigned diagnosis is in doubt. As with the case-finding application, more efficient use of spirometry is likely to result. However, enhanced diagnostic accuracy may provide additional benefit by improving the overall management of patients with these diseases. The expected probability of fixed obstruction in this group is approximately 50%, which should generate a spirometry referral in most developed countries.
Limitations
Our response rates were fairly low (24.2% in Aberdeen and 6.1% in Denver). Since we mailed to a random sample of all persons > 40 years old irrespective of smoking or lung disease history, we cannot determine exactly how many of the persons who failed to respond were actually ineligible for the study. It is difficult to know how nonresponse bias might have affected our primary study question of which questions are associated with obstruction. At any rate, our subjects volunteered to participate and do not necessarily reflect the general nature of the primary care practice population, though they may represent persons likely to come forward for screening in primary care.
Our questionnaires were tested and validated using different subsets drawn from the same population. Despite the splitting of the samples to reduce same-sample bias, other potential sources of bias could have affected the performance of the final instruments. Systematic differences in diagnostic or prescribing patterns, for example, would affect the assignment of patients into the case-finding or differential diagnosis sample. Our sample size was quite large relative to most previous studies of respiratory diagnostic questionnaires; nevertheless, we found some instability in the regression coefficients and variance estimates between development and performance subsamples. This may represent idiosyncratic differences between these two subsamples or unmeasured systematic differences. Resolution of these questions requires validation in an independent population sample.
We made our study diagnoses based on spirometry performed during a single visit. This is an epidemiologic case-definition approach and thus represents an oversimplification of clinical diagnosis. It is possible that some proportion of patients might have received a different study diagnosis had they been subjected to a full diagnostic workup, including a course of steroid therapy. Although most of the question items included in the final questionnaires were derived from previously validated instruments, test-retest reliability of the final instruments should be performed.
Finally, different clinical populations are likely to have different baseline prevalences of airway obstruction. Since the questionnaire scoring system is based on positive and negative predictive values (which are sensitive to prevalence), questionnaire performance is likely to be different in these populations. The PPV increases as prevalence increases while the NPV decreases with increasing prevalence, although most of the changes in performance parameters are generally small within the ranges likely to be seen in clinical practice. For example, if we administered the case-finding questionnaire to persons with a baseline obstruction prevalence of 30% (50% higher than observed in our results), we would expect the questionnaire to have a higher PPV (52%) at cut point A and a slightly lower NPV (87%) at cut point B. This translates into an improved confidence in referring for spirometry and slightly decreased confidence in deferring this test. If we administered the differential diagnosis questionnaire to a population with the same prevalence of 30%, we would expect a lower PPV (66%) at cut point A leading to decreased confidence in spirometry referral, and a slightly higher NPV (89%) at cut point B leading to increased confidence in deferring spirometry. For these reasons, administering these questionnaires using modified selection criteria (for example, using the questionnaires in an unselected general population, or using different age thresholds) is not supported by these results.
Conclusions
The questionnaires described here can be used to identify persons likely to have COPD among specific risk groups. This can be done with acceptable performance characteristics. The use of a simple scoring system makes these tools beneficial in the primary care setting. Used in conjunction with spirometry, these tools can help improve the efficiency and accuracy of COPD diagnosis in primary care. Future work should address the reliability of the instruments, validity in different populations and practice environments, and the cost impact of implementing these tools.
Appendix
Members of the COPD Questionnaire Study Group include William Erhardt, MD, Pfizer Inc, New York, NY; Daryl Freeman, MD, University of Aberdeen, Aberdeen, Scotland; R. J. Halbert, MD, MPH, Cerner Health Insights, Beverly Hills, CA; Thomas Hausen, MD, Essen, Germany; Sharon Isonaka, MD, MS, Cerner Health Insights, Beverly Hills, CA; Elizabeth F. Juniper, MSc, McMaster University, Hamilton, ON, Canada; Claus Justus, DVM, Boehringer Ingelheim International GmbH, Ingelheim, Germany; Mark L. Levy, MD, University of Edinburgh, Edinburgh, Scotland; Dmitry Nonikov, MD, Wiesbaden, Germany; Robert J. Nordyke, PhD, Cerner Health Insights, Beverly Hills, CA; Anders Østrem, MD, Oslo, Norway; David B. Price, MD, University of Aberdeen, Aberdeen, Scotland; David G. Tinkelman, MD, National Jewish Medical and Research Center, Denver, CO; Thys van der Molen, MD, University of Groningen, Groningen, the Netherlands; and Constant P. van Schayck, PhD, University of Maastricht, Maastricht, the Netherlands.
Acknowledgements
The authors thank the subjects who participated in the study; the staff who collected the data; study coordinators Jan Caldow in Aberdeen and Melanie Gleason in Denver; Dr. John L. Adams, who provided statistical advice; and Dr. Michael Levine, who served as a spirometry reviewer.
Footnotes
Abbreviations: NPV = negative predictive value; PPV = positive predictive value; ROC = receiver operator characteristic
This project was funded by Boehringer Ingelheim International GmbH and Pfizer Inc.
Portions of this article were presented at meetings of the International Primary Care Respiratory Group (International Congress: Melbourne, Australia; February 19 to 22, 2004) and the Asian Pacific Society of Respirology (Ninth Congress: Hong Kong, December 10 to 13, 2004).
Dr. Price has received honoraria for speaking at sponsored meetings and serving on advisory panels for the following companies marketing COPD products: AstraZeneca, Boehringer Ingelheim, GlaxoSmithKline, Novartis, and Pfizer. He or his research team have received funding for research projects from the same companies. Dr. Tinkelman is an employee of National Jewish Medical and Research Center, which received funding from Boehringer Ingelheim to participate in this study. At the time this work was performed, Drs. Nordyke, Isonaka and Halbert were employees of Cerner Health Insights, which provides consulting services to the pharmaceutical industry, including the sponsors of this project.
Received for publication October 3, 2005. Accepted for publication December 3, 2005.
References
This article has been cited by other articles:
![]() |
P. Enright and P. Quanjer Spirometry for COPD Is Both Underutilized and Overutilized Chest, August 1, 2007; 132(2): 368 - 370. [Full Text] [PDF] |
||||
![]() |
D. Freeman and D. Price Primary care and palliative care BMJ, July 22, 2006; 333(7560): 188 - 190. [Full Text] [PDF] |
||||
Read all eLetters
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |