|
|
||||||||
Guest Access | Sign In via User Name/Password |
|||||||||
* From the Department of Respiratory Medicine and Clinical Physiology (Drs. Eaton, Garrett, and Whitlock, and Mr. Withy), Green Lane Hospital, Auckland, New Zealand; and Department of General Medicine (Dr. Rea), Middlemore Hospital, Auckland, New Zealand.
Correspondence to: T. Eaton, MBChB, Department of Respiratory Services, Green Lane Hospital, Auckland 3, New Zealand
| Abstract |
|---|
|
|
|---|
Design: Randomized, controlled prospective interventional study.
Setting: Primary care practice, Auckland City, New Zealand.
Participants: Thirty randomly selected primary care practices randomized to "trained" or "usual" groups. One doctor and one practice nurse were nominated to participate from each practice.
Interventions: "Trained" was defined as participation in an "initial" spirometry workshop at week 0 and a "maintenance of standards" workshop at week 12. "Usual" was defined as no formal training until week 12, when participants they attended the same "initial" workshop provided for the trained group. The study duration was 16 weeks. Each practice was provided with a spirometer to be used at their clinical discretion.
Measurements and results: Spirometry data were uploaded weekly and analyzed using American Thoracic Society (ATS) criteria for acceptability and reproducibility. The workshops were assessed objectively with practical and written assessments, confirming a significant training effect. However, analysis of spirometry performed in clinical practice by the trained practitioners revealed three acceptable blows in only 18.9% of patient tests. In comparison, 5.1% of patient tests performed by the usual practitioners had three acceptable blows (p < 0.0001). Only 13.5% of patient tests in the trained group and 3.4% in the usual group (p < 0.0001) satisfied full acceptability and reproducibility criteria. However, 33.1% and 12.5% of patient tests in the trained and usual groups, respectively (p < 0.0001), achieved at least two acceptable blows, the minimum requirement. Nonacceptability was largely ascribable to failure to satisfy end-of-test criteria; a blow of at least 6 s. Visual inspection of the results of these blows as registered on the spirometer for the presence of a plateau on the volume-time curve suggests that < 15% were acceptable.
Conclusions: Although a significant training effect was demonstrated, the quality of the spirometry performed in clinical practice did not generally satisfy full ATS criteria for acceptability and reproducibility. Further study would be required to determine the clinical impact. However, the ATS guidelines allow for the use of data from unacceptable or nonreproducible maneuvers at the discretion of the interpreter. Since most of the failures were end-of-test related, the FEV1 levels are likely to be valid. Our results serve to emphasize the importance of effective training and quality assurance programs to the provision of successful spirometry in primary care practice.
Key Words: primary care practice quality assurance randomized controlled spirometry spirometry workshops
| Introduction |
|---|
|
|
|---|
Although spirometry is often described as a simple screening test, due consideration is essential not only of equipment selection, but, importantly, of test performance and correct interpretation of the results. In 1991, the American Thoracic Society (ATS) Statement on Lung Function Testing stated: "The largest single source of within subject variability is improper performance of the test."8 This was persuasively addressed in the Lung Health study, particularly with regard to the importance of ongoing maintenance of standards.9 10 Hence, effective training and quality assurance are vital prerequisites for successful spirometry.11 While well-established criteria for acceptability and reproducibility have been widely disseminated, it is by no means certain that these are adhered to in clinical practice. Excepting research studies and accredited pulmonary function laboratories,12 there are no formal quality assurance programs in place. Quality assurance is crucial to prevent misleading results and misdiagnoses. If spirometry is to be promoted as a screening tool in primary care practice, it is important that careful attention is paid to ensuring that quality standards are met.
No previous study has formally assessed spirometry performance in primary care practice. The development of "smart" spirometers has enabled the quality of spirometry performed in clinical practice to be objectively assessed using the ATS criteria for acceptability and reproducibility.11 We aimed to determine both the quality and the impact of training on spirometry performed in primary care practice.
| Materials and Methods |
|---|
|
|
|---|
Three separate evaluations were performed:
Study Design
As shown in Figure 1
, practices were randomly assigned to spirometry training ("trained"
group, n = 15) or to the performance of spirometry without prior
instruction ("usual" group, n = 15).
|
In the usual group, the spirometer was delivered to the doctor and nurse with instructions for its operation, but no training in spirometry performance was given. At week 12, they attended the same spirometry workshop provided in week 0 for the trained group.
Spirometry Workshops
The initial workshop was 2 h in duration and comprised
theoretical and practical aspects of spirometry performance, with
particular attention paid to acceptability and reproducibility criteria
and the importance of quality assurance (see "Appendix").
The spirometry workshops were led by the clinical director of our
pulmonary function laboratory, two consultant pulmonologists, a
clinical respiratory scientific officer, and charge respiratory
technician. All had > 15 years of experience in the field. This
workshop was held at week 0 for the trained group and week 12 for the
usual group. The trained group received a 90-min "maintenance of
standards" workshop at week 12. Quality assurance was emphasized.
Following the workshops, the results of the written and practical assessments and feedback on the spirometry they had performed in the previous 12 weeks (provisionally analyzed for acceptability and reproducibility) were discussed individually with each practitioner.
Spirometers
Each practice was provided with a handheld spirometer (2120 U;
Vitalograph Limited; Buckingham, UK) certified by Dr. R.O.
Crapo, MD (Medical Director of the Pulmonary Laboratory at LDS
Hospital, Salt Lake City, UT) as meeting ATS standards.11
This unit uses a pneumotachograph flow sensor. A perceived
advantage of the unit was the provision of "built-in" quality
assurance features, based on ATS criteria.11
Prompts were
displayed after each blow for an unacceptable blow (eg, slow
start, or abrupt end), for recommending the performance of three or
more blows, and giving the variability between the two largest values
for FEV1 and for FVC.
Data Collection
During the study period, only the nominated doctor and/or
practice nurse from each practice used the spirometer. Both groups were
provided with the clinical indications for spirometry.13
They were advised to use the spirometer entirely at their clinical
discretion. The spirometer had a 100-test memory, allowing data to be
uploaded at weekly intervals; in this way any study effect on the
generation of spirometry by the participants was minimized. A unique
patient identification code was entered. Standard demographic
data (age, sex, height, and ethnicity) were entered with the derivation
of normal predicted values.14
15
For study purposes, only
expiratory parameters were measured: FEV1, FVC,
peak expiratory flow rate, and forced expiratory flow (midexpiratory
phase).
Primary Outcome Assessments
Spirometry Quality Assurance: All the spirograms generated
by the practitioners during the study period were analyzed for
acceptability and reproducibility as specified by the ATS quality
criteria.11
Acceptability Criteria: Individual spirograms were judged
"acceptable" if all of the following were satisfied: good start (as
defined by an extrapolated volume < 5% of FVC or 150 mL, whichever
is greater); satisfactory exhalation for
6 s; free from abrupt end;
and free from cough.
Reproducibility Criteria: After three acceptable spirograms were identified, the following tests were applied. Were the two largest FVC values within 200 mL of each other? Were the two largest FEV1 values within 200 mL of each other? If both these criteria were satisfied, the test was judged reproducible.
"Good start" has also been defined by time-to-peak expiratory flow (PEF). Time-to-PEF is not currently a standard recommendation in the ATS criteria11 and, hence, was not used in the overall analysis of acceptability. However, the spirometry unit used in this study incorporated quality prompts with poor start (defined as a time-to PEF > 85 ms), as did the Lung Health study.9
Since blows of < 6 s may be acceptable if a plateau is reached, a random selection of these blows was scrutinized by two experienced pulmonologists. All blows < 4 s were automatically judged nonacceptable. A plateau was defined as no visible change in volume for at least 1 s.
Practical and Written Spirometry Assessments: Both groups were formally assessed on practical performance of spirometry at week 12 (see "Appendix"). Each practitioner was asked to perform spirometry on a "naive subject" and was scored on a scale of 1 to 10 by one of five trained examiners. Although the examiners were not blinded, scoring bias was minimized by using stringent objective criteria. A score of 8 was judged "acceptable." The trained group completed a written assessment before and immediately after the week 0 workshop. The same assessment was repeated before the week 12 workshops in both groups. The assessment included only material presented at the workshop. We developed a video of "poorly performed" spirometry containing five errors to be identified. The written and practical assessments were fed back to the trained group at week 12 (ie, the formative assessment).
Indications for and Interpretation of Spirometry: At the end of the study, each practice was provided with 25 patient identifications, randomly selected from their patients who had performed spirometry in the preceding 16 weeks. Patient demographic data were recorded, and the doctor was asked to specify the indication for spirometry (eg, screening of an asymptomatic smoker). For each spirometric record, the doctor was asked to provide an interpretation, choosing from the following possibilities: normal, early small airways disease, obstruction, restriction, mixed obstruction/restriction, inability to interpret due to inadequate spirometry, and other. These records were then reviewed by two experienced pulmonologists. The same information was provided as was available to the primary care physician; patient demographic data, "indication," forced expiratory indexes with normal predicted values, the expiratory curve, and the acceptability and reproducibility prompts. Primary care physician interpretations were then marked as "correct" or "incorrect."
Statistical Analysis
Data were normally distributed and were presented as mean (SD).
Proportions were compared by Fisher's Exact Test. Changes in scores
were analyzed using paired Student's t test. Logistic
regression was used to explore predictors of "acceptable"
spirometry. A p value < 0.05 was considered significant. All analyses
were performed on a personal computer (IBM-compatible) using
appropriate software (SAS, version 6.1 for Windows; SAS Institute;
Cary, NC).
| Results |
|---|
|
|
|---|
Spirometry Quality Assurance
Spirometry quality for weeks 0 to 11 is shown in Table 1
. Patient tests satisfied ATS criteria for acceptability and
reproducibility more frequently in the trained group than in the usual
group (p < 0.0001). Nonacceptability was largely ascribable to
failure to satisfy end-of-test criteria (Table 2
). From a total of 2,928 blows, 825 (28%) were
6 s, and 1,380
(47%) were < 4 s and were rejected automatically. The remaining 723
blows (25%) of 4 to < 6 s duration may have been acceptable
on further scrutiny. Visual inspection of 245 blows (33%) demonstrated
a plateau in 90 (37%). This finding suggests that < 15% of blows
that were judged unacceptable on the grounds of inadequate duration
were acceptable. Individual blows were more often unacceptable in the
usual group (p < 0.0001); however, following the week 12 workshop, a
clear training effect was observed (p < 0.0001) (Fig 2
). The proportion of acceptable blows was then consistent with the
trained group. Lung function and spirometry quality assurance data were
tabulated by age (Table 3
). On logistic regression, the major determinant of an acceptable
maneuver was "training" (p = 0.0001). Blows were more often
unacceptable at the extremes of age (< 10 years and > 80 years;
p = 0.01), in women (p = 0.05), and in non-Europeans (p = 0.006).
|
|
|
|
|
|
| Appendix: Program For Spirometry Workshops |
|---|
|
|
|---|
1. Brief review of first principles.
2. Definitions of the variables to be measured: FEV1, FVC, peak expiratory flow rate, forced expiratory flow (midexpiratory phase) with labeling of the spirogram and flow-volume curves.
3. Physiology of flow-volume curves.
4. Definition of ambient temperature pressure saturated and body temperature pressure saturated.
5. Derivation of normal predicted values.
6. Interpretation of results: normal, early small airways disease, obstruction, restriction.
7. Indications for spirometry.
8. Quality issues:
a. the importance of obtaining good-quality spirometry;
b. definitions of acceptability and reproducibility (ATS criteria11 ); and
c. examples of expiratory flow curves: poor quality vs good quality.
9. Performance of spirometry (theory): postbronchodilator testing.
10. Demonstration of spirometry unit: maintenance, calibration, and transmission of infection issues.
11. Practical experience: entry of patient identification, demographic data, and performance of spirometry (based on ATS performance criteria11 ).
Maintenance of Standards
1. Quality assurance: revision of the importance of quality assurance; individual feedback on the quality of their spirometry from the preceding 12 weeks.
2. Interpretation of 16 representative flow-volume curves (30 min).
3. Practical performance of spirometry: basic revision plus individual feedback from each individual's practical spirometry assessment ( and ).
| Discussion |
|---|
|
|
|---|
There is a danger of undue negativity based on an uncritical "interpretation" of our data. The ATS statement allows for the use of data from unacceptable or nonreproducible maneuvers at the discretion of the interpreter. We appreciate that a blow of < 6 s may be acceptable if a plateau has been achieved. Further analysis suggested that this was only achieved in a small proportion of blows. It is likely that primary care practitioners, practice nurses, and, indeed, the patients have been well schooled in the use of a peak flow meter for asthma. Since asthma was given as the indication for testing in 40% of cases, this may be a possible explanation for the relatively "good starts" but unacceptably "short" blows. However, since the majority of test failures were end-of-test related, the FEV1 level is likely to be valid and could be used for serial monitoring. Failure to meet the ATS criteria does not necessarily imply that the test performance was clinically invalid or unusable. Satisfaction of rigid criteria is likely to be crucial only when abnormalities are marginal. In most cases, results are likely to be interpretable if values are within normal limits or are substantially deranged. However, minor abnormalities should be interpreted with great care.
Our spirometry workshops were successful, as judged by the traditional performance criteria of written and practical assessments. These assessments were incorporated into our workshops both to reinforce educational goals and to provide objective measures of the efficacy of our training. The preworkshop written assessment exposed low baseline knowledge. Immediately postworkshop, the scores improved significantly, albeit representing short-term recall only. The results of retesting at week 12 demonstrated the need for continuing education to maintain standards. Our training program was very consistent in the magnitude of its training effect. Following week 12, when the usual groups had received the "initial" spirometry workshop given to the trained group at week 0, both groups were achieving a very similar proportion of acceptable blows. The practical assessments also demonstrated a significant training effect. However, although statistically significant, the improvement associated with training may not have been of clinical value. An acceptable practical assessment did not necessarily translate into acceptable spirometry when performed in clinical practice. Prior to the development of "smart" spirometers, we did not have the ability to objectively assess spirometry performed in clinical practice. It is perhaps not unexpected that the results were appreciably different from those obtained using more traditional assessments. It is well recognized that knowledge may not reliably predict behavior.16
Although we primarily addressed the quality of spirometry performance, the interpretations of spirometry were incorrect in almost 50% of the cases reviewed. Accurate interpretation is highly dependent not only on a well-performed procedure, but also on an appreciation of physiology, appropriate choice of normal values, and clinical knowledge of the patient. Our results indicate an important gap in knowledge and understanding that clearly requires more training and experience than our workshops could provide.
It is certainly possible that longer, more intensive workshops may have produced better results. However, if spirometry is to be widely available in primary care practice, the sheer logistics of training and maintaining standards among large numbers of practitioners dictates a condensed and pragmatic training program. Our workshops did pay particular attention to the acceptability and reproducibility criteria and to the importance of quality assurance. Following the workshops, all practitioners at week 12 also received individual feedback on their written and practical assessments. Most importantly, we thought, they were given individual feedback on the quality of the spirometry they had performed in the previous 12 weeks, which had been provisionally analyzed for acceptability and reproducibility. The Lung Health Study demonstrated that, even in a dedicated research setting with meticulous attention to quality, technician performance fell over time.9 However, the quality of spirometry not only improved dramatically, but was maintained with regular monitoring of test session quality and prompt individual feedback. We selected the Vitalograph 2120 spirometer with its inbuilt quality assurance prompts with the expectation that it would improve the quality of spirometry. This, unfortunately, did not appear to be the case, although, to our knowledge, a comparison of performance quality between spirometers with and without quality prompts has not been done.
Even when a spirometer is available on-site, underuse remains a problem, as demonstrated by a Canadian study.17 Despite almost 60% of doctors having direct access to spirometry equipment, primary care doctors had a low index of suspicion for COPD and markedly underused spirometry. In our study, the mean number of patient tests per week was only 2.3. The reasons for this are likely to be multifactorial and may change with further education and the use of incentives. Acknowledging the recent National Lung Health Education Program publication,1 it is disappointing that, despite education, spirometry was clearly underutilized in the early diagnosis of COPD and in supporting smoking cessation strategies, where it has a crucial role.18 19 20 The public health implications for appropriately targeted smoking cessation programs using screening spirometry are very promising.
Recent British Thoracic Society guidelines for COPD21 acknowledge that health planners may need to consider options for the provision of spirometry in primary care practice other than having the appropriate equipment on-site. The provision of spirometry, where quality issues can be addressed and maintained, may only be achieved by limiting spirometry to a smaller number of community clinics or pulmonologists or by increasing access to pulmonary function laboratories where quality-control measures should already be in place.12
However, ideally, spirometry would be available on-site in the primary care practice. Our results serve as a reminder of the importance of effective training and quality-assurance programs in the provision of successful spirometry.
|
|
| Acknowledgements |
|---|
| Footnotes |
|---|
For editorial comment see page 276.
Abbreviations: ATS = American Thoracic Society; PEF = peak expiratory flow
Received for publication June 29, 1998. Accepted for publication March 23, 1999.
| References |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
B. P. Yawn, P. L. Enright, R. F. Lemanske Jr, E. Israel, W. Pace, P. Wollan, and H. Boushey Spirometry Can Be Done in Family Physicians' Offices and Alters Clinical Decisions in Management of Asthma and COPD Chest, October 1, 2007; 132(4): 1162 - 1168. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. N. Aggarwal, D. Gupta, and S. K. Jindal The relationship between FEV1 and peak expiratory flow in patients with airways obstruction is poor. Chest, November 1, 2006; 130(5): 1454 - 1461. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. J P Poels, T. R J Schermer, C. van Weel, and P. M A Calverley Spirometry in chronic obstructive pulmonary disease. BMJ, October 28, 2006; 333(7574): 870 - 871. [Full Text] [PDF] |
||||
![]() |
D M Mannino Spirometric screening: does it work? Thorax, October 1, 2006; 61(10): 834 - 835. [Full Text] [PDF] |
||||
![]() |
G. Liistro, C. Vanwelde, W. Vincken, J. Vandevoorde, G. Verleden, J. Buffels, and on Behalf of the COPD Advisory Board Technical and functional assessment of 10 office spirometers: a multicenter comparative study. Chest, September 1, 2006; 130(3): 657 - 665. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. E. Hansen, X-G. Sun, and K. Wasserman Should forced expiratory volume in six seconds replace forced vital capacity to detect airway obstruction? Eur. Respir. J., June 1, 2006; 27(6): 1244 - 1250. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. Zielinski, M. Bednarek, D. Gorecka, G. Viegi, S. S. Hurd, Y. Fukuchi, C. K. W. Lai, P. X. Ran, F. W. S. Ko, S. M. Liu, et al. Increasing COPD awareness. Eur. Respir. J., April 1, 2006; 27(4): 833 - 852. [Full Text] [PDF] |
||||
![]() |
M. Lusuardi, F. De Benedetto, P. Paggiaro, C. M. Sanguinetti, G. Brazzola, P. Ferri, and C. F. Donner A Randomized Controlled Trial on Office Spirometry in Asthma and COPD in Standard General Practice : Data From Spirometry in Asthma and COPD: a Comparative Evaluation Italian Study,{dagger}. Chest, April 1, 2006; 129(4): 844 - 852. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. Pellegrino, M. Decramer, C. P. O. van Schayck, P. N. R. Dekhuijzen, T. Troosters, C. van Herwaarden, D. Olivieri, M. Del Donno, W. De Backer, I. Lankhorst, et al. Quality control of spirometry: a lesson from the BRONCUS trial Eur. Respir. J., December 1, 2005; 26(6): 1104 - 1109. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. Zanconato, G. Meneghelli, R. Braga, F. Zacchello, E. Baraldi, and on behalf of the Working Group Office Spirometry in Primary Care Pediatrics: A Pilot Study Pediatrics, December 1, 2005; 116(6): e792 - e797. [Abstract] [Full Text] [PDF] |
||||
![]() |
O. C. Ioachimescu, S. B. Venkateshiah, M. S. Kavuru, K. McCarthy, and J. K. Stoller Estimating FVC From FEV2 and FEV3: Assessment of a Surrogate Spirometric Parameter Chest, September 1, 2005; 128(3): 1274 - 1281. [Abstract] [Full Text] [PDF] |
||||
![]() |
B G Cooper Limitations to spirometry being performed in 'the office' Chronic Respiratory Disease, April 1, 2005; 2(2): 113 - 115. [PDF] |
||||
![]() |
M Decramer, R Gosselink, M Rutten-Van Molken, J Buffels, O Van Schayck, P-A Gevenois, R Pellegrino, E Derom, and W De Backer Assessment of progression of COPD: report of a workshop held in Leuven, 11-12 March 2004 Thorax, April 1, 2005; 60(4): 335 - 342. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. A. Kaminsky Spirometry and Diabetes: Implications of reduced lung function Diabetes Care, March 1, 2004; 27(3): 837 - 838. [Full Text] [PDF] |
||||
![]() |
S. Kim, C. L. Emerman, R. K. Cydulka, B. H. Rowe, S. Clark, and C. A. Camargo Prospective Multicenter Study of Relapse Following Emergency Department Treatment of COPD Exacerbation Chest, February 1, 2004; 125(2): 473 - 481. [Abstract] [Full Text] [PDF] |
||||
![]() |
T. Schermer, T. Eaton, R. Pauwels, and C. van Weel Spirometry in primary care: is it good enough to face demands like World COPD Day? Eur. Respir. J., November 1, 2003; 22(5): 725 - 727. [Full Text] [PDF] |
||||
![]() |
T R Schermer, J E Jacobs, N H Chavannes, J Hartman, H T Folgering, B J Bottema, and C van Weel Validity of spirometric testing in a general practice population of patients with chronic obstructive pulmonary disease (COPD) Thorax, October 1, 2003; 58(10): 861 - 866. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. Golshan, M. Nematbakhsh, B. Amra, and R.O. Crapo Spirometric reference values in a large Middle Eastern population Eur. Respir. J., September 1, 2003; 22(3): 529 - 534. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. Miravitlles, C. Murio, T. Guerrero, and R. Gisbert Costs of Chronic Bronchitis and COPD: A 1-Year Follow-up Study Chest, March 1, 2003; 123(3): 784 - 791. [Abstract] [Full Text] [PDF] |
||||
![]() |
I. Cerveri, A. Corsico, M. C. Zoia, J. Zielinski, and M. Bednarek COPD Screening in High-Risk Groups Chest, March 1, 2003; 123(3): 959 - 960. [Full Text] [PDF] |
||||
![]() |
M. Miravitlles, C. Murio, T. Guerrero, and R. Gisbert Pharmacoeconomic Evaluation of Acute Exacerbations of Chronic Bronchitis and COPD* Chest, May 1, 2002; 121(5): 1449 - 1455. [Abstract] [Full Text] [PDF] |
||||
![]() |
T. L. Petty Commentary: Quality of Spirometry Testing American Journal of Medical Quality, November 1, 2001; 16(6): 216 - 218. [Abstract] [PDF] |
||||
![]() |
M. Miravitlles, C. Murio, and T. Guerrero Factors associated with relapse after ambulatory treatment of acute exacerbations of chronic bronchitis Eur. Respir. J., May 1, 2001; 17(5): 928 - 933. [Abstract] [Full Text] [PDF] |
||||
![]() |
G. T. Ferguson, P. L. Enright, A. S. Buist, and M. W. Higgins Office Spirometry for Lung Health Assessment in Adults: A Consensus Statement From the National Lung Health Education Program Chest, April 1, 2000; 117(4): 1146 - 1161. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||