| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
Guest Access | Sign In via User Name/Password
|
|
|
|||||||
Guest Access | Sign In via User Name/Password |
|||||||||
* From the Pneumology Unit (Dr. Liistro), Cliniques Universitaires Saint-Luc, Brussels; Department of General Practice (Dr. Vanwelde), Université Catholique de Louvain, Louvain-la-Neuve; Pneumology Unit (Dr. Vincken) and Department of General Practice (Dr. Vandevoorde), Vrije Universiteit Brussel, Brussels; and Pneumology Unit (Dr. Verleden) and Department of General Practice (Dr. Buffels), Katholieke Universiteit Leuven, Leuven, Belgium.
Members of the COPD Advisory Board are given in the Appendix.
Correspondence to: Giuseppe Liistro, MD, PhD, Pneumology Unit, Cliniques Universitaires Saint-Luc, Avenue Hippocrate 10, 1200 Bruxelles, Belgium; e-mail: Giuseppe.liistro{at}clin.ucl.ac.be
Abstract
Study objectives: To investigate the technical properties and user friendliness of 10 office spirometers devoted for use in general practice, and to compare the results with standard diagnostic spirometers.
Design: Multicenter study.
Setting: Ten spirometer models were tested independently in three pulmonary function laboratories and by three general practitioners (GPs).
Measurements: The laboratories studied the technical quality of the office spirometers in terms of precision and agreement with standard spirometers, whereas the three GPs assessed their user friendliness. The spirometers tested were as follows: Spirobank (Medical International Research; Rome, Italy); Simplicity (Puritan Bennett; Pleasanton, CA); OneFlow (Clement Clarke International; Harlow, Essex, UK); Datospir 70 (Sibelmed; Barcelona, Spain); Datospir 120 (Sibelmed); SpiroPro (SensorMedics; Yorba Linda, CA); EasyOne (NDD; Zurich, Switzerland); MicroLoop (Micro Medical; Chatham, Kent, UK); SpiroStar (Medikro; Kuopio, Finland); and Pneumotrac (Vitalograph; Maids Moreton, Buckingham, UK). FVC and FEV1 were measured in 399 subjects. User friendliness was assessed by the three GPs using a questionnaire.
Results: The precision of FEV1 of the office and standard spirometers was comparable, but three office spirometers had > 200 mL limits of precision for FVC. Some devices presented a proportional difference on the FEV1 with standard spirometers, underestimating the small values. The limits of agreements between standard and some office spirometers for FEV1/FVC ratio was > 10%. The overall user friendliness was estimated as good.
Conclusions: The global quality and user friendliness of several office spirometers make them acceptable for the detection of COPD, although differences between the laboratory and some of the office spirometers values suggest that the misclassification rates may be increased when using some models of office spirometers.
Key Words: comparative study • COPD detection • pulmonary function tests • spirometer • spirometry
There is a clear need for early diagnosis of COPD.1 This lung disease is one of the leading causes of mortality and disability in developed countries, and only smoking cessation has proven its efficacy in changing the natural evolution of COPD.2 One major problem with early detection of COPD is the fact that smokers rarely complain even if they have dyspnea. However, lung function changes are often detectable > 10 years before onset of dyspnea at rest.3 Therefore, according to a consensus statement from the National Lung Health Education Program,4 the screening of asymptomatic at-risk populations should start from the age of 45 years. The screening by general practitioners (GPs) using office spirometry can double the number of early diagnoses in COPD patients.5 Therefore, primary care providers should be encouraged to perform good quality spirometry. For that, a good spirometer is as important as good training. Previous studies678910 were conducted to assess the quality of some hand-held spirometers. These studies678910 usually compared one small spirometer with a conventional spirometer, and some found significant differences between the devices. However, small electronic spirometers constantly improve, and it is difficult to have a precise opinion on the quality of all the models present on the market. Moreover, the poor technical quality of some office spirometers may be an obstacle for routine clinical use and for the interchangeability of the measurements.6 According to their manufacturers, the majority of the modern office spirometers do not require a daily calibration check. If this were true, it would represent an advantage because calibration checks are seldom carried out in general practice.11 The aims of the present study were to assess the technical properties and the user friendliness of 10 spirometers devoted for use in general practice, and to compare the results with standard diagnostic spirometers.
Materials and Methods
In 2002, we asked the sales representatives of office spirometers available in Belgium to propose one or two models for use in general practice. The sales representatives of the office spirometers were first contacted through GlaxoSmithKline Belgium. Table 1 presents the 10 devices tested, their type of flow sensor, and their manufacturers.
|
For the user-friendliness assessment, the office spirometers were also presented to three GPs working in the general practice department of the three universities. The same office spirometers were used successively in the three laboratories and by the three GPs according to their availability.
Methods
The sales representatives of the office spirometers were asked to demonstrate the devices in each center. At the moment of the study, Sibelmed (Barcelona, Spain) was not represented in Belgium and the two Sibelmed spirometers (Datospir 70 and Datospir 120) were not demonstrated. We scrupulously followed the instructions of the manufacturers, in particular concerning the handling of the devices and the need for calibration checks. According to instruction manuals, the majority of the office spirometers do not need calibration.
In-Laboratory Study
The 10 office spirometers were tested independently in the three laboratories following the same protocol. The office spirometers were compared to standard diagnostic spirometers: Vmax 20C (SensorMedics; Bilthoven, the Netherlands; software: 5/2A, 2002) in two centers and Morgan TLC (Morgan Medical; Rainham, UK; software: Mdas 4.01, 1999) in one center. These devices are calibrated daily and used by expert technicians (ETs) only. The Vmax 20C, a flow-sensing spirometer, was calibrated with a 3-L syringe at three different speeds; the Morgan TLC, a volume-sensing device, using a single speed, was calibrated as recommended by the manufacturer. The Morgan TLC was checked daily for leaks. A log of calibration results and leaks checks was maintained.
In each laboratory, three ETs tested all the office spirometers. We first checked the ability of the ETs themselves to perform reproducible pulmonary function tests (PFTs), and the interchangeability of the results between the three centers. Only technicians able to blow five times successively with maximum variations of FEV1 or FVC of 200 mL or 6% were selected. All were nonasthmatic nonsmokers and were free of respiratory symptoms. The ETs were asked to perform PFTs in the two other centers to verify interchangeability of the results. There was no significant difference between centers in absolute values and reproducibility (p > 0.05, by one-way analysis of variance; Fig 1 ).
|
Agreement Between the Standard and Office Spirometers: In each center, PFTs were performed by healthy naive subjects and 10 COPD patients with the office spirometers and with the standard spirometers. The healthy subjects were nonsmokers, free of any respiratory symptoms, and members of the hospital staff. These tests were done in a random order to avoid a learning effect. We asked the three centers to select the patients to represent various degrees of severity of COPD (Global Initiative for Chronic Obstructive Lung Disease [GOLD] stages I to IV). We retained only the patients and subjects able to perform the PFT according to American Thoracic Society (ATS) quality criteria.12 We compared the best values of FVC, FEV1, and FEV1/FVC from all acceptable tests for all pairs of devices (each office spirometer vs standard spirometer).
Because of the multicentric nature of the study and the fact that the same devices were tested successively in each center, the healthy volunteers and the COPD patients were different for each apparatus and by center. Each office spirometer was tested with 48 subjects ([three ETs + 3 healthy subjects + 10 COPD patients] x 3), a sample size comparable to previous studies.910
Assessment of User Friendliness
We developed a novel questionnaire to assess the user friendliness of office spirometers (to view Tables A and B in the on-line supplementary data, go to www.chestnet.org). The questionnaire completed by the three GPs covered the following: general properties and parameters of the software, quality of the patient administrative data, features of the display and automated quality control, comparison of successive tests in the same subject, use at home visits, and export facilities.
Analysis
The precision of the office spirometers was assessed by the Sw of FEV1 and FVC obtained from five successive maneuvers done by the nine ETs with the office spirometers and with the standard spirometers.13 The larger the Sw value, the lower the precision. The variance of each set of five measurements (FVC, FEV1) was computed, and the within-subject variance of the measurements was obtained by averaging the nine variances. To obtain the 95% error limits, the square root of the within-subject variance was multiplied by 1.96. We used goals for within-session repeatability for FEV1 and FVC of 200 mL, as our upper limits of precision.
The preliminary analyses revealed no difference between the populations studied, so the results of the PFTs done in the three centers were pooled for each office spirometer. The agreement and the bias between the standard and office spirometers were examined using Bland and Altman14 analysis. The bias (mean difference between the office spirometers and the standard spirometers), its 95% confidence interval (CI), and the lower and upper limits of agreement between office spirometers and standard spirometers were reported. A statistical significant correlation (p < 0.05) indicates the presence of a proportional difference between the devices. We fixed the upper limits of acceptable bias at ± 100 mL according to the ATS accuracy criteria for diagnostic devices.12 The acceptable limits of agreement between the office spirometers and the standard spirometers were 350 mL and 500 mL for FEV1 and FVC, respectively. These limits were fixed according to the short-term coefficient of variability of FEV1 and FVC measured in COPD patients.15
Results
In-Laboratory Study
A total of 399 different subjects (age range, 25 to 87 years; mean ± SD, 61.2 ± 14.6 years; 128 women) were studied in the three centers. The distribution of the 300 COPD patients according to the GOLD classification was as follows: stage 1, 10.3%; stage 2, 20.4%; stage 3, 59.0%; and stage 4, 6.3%. The nine ETs (three women) all had normal spirometric data (FVC, 4.68 ± 1.04 L; FEV1, 3.80 ± 0.90 L).
Table 2 presents the precision of the standard diagnostic spirometers and each model of office spirometer. The precision was measured in ETs from the three centers (n = 9). The precision of FEV1 was comparable between the standard and office spirometers, except for one device (Simplicity; Puritan Bennett; Pleasanton, CA), in which the limits of precision were > 200 mL. Three models—OneFlow (Clement Clarke International; Harlow, Essex, UK), EasyOne (NDD; Zurich, Switzerland), and Simplicity—showed precision limits > 200 mL for FVC.
|
100 mL). The limits of agreement were different between the office spirometers according to the variable studied. Only three devices had acceptable limits of agreement for FVC (MicroLoop; Micro Medical; Chatham, Kent, UK; Pneumotrac; Vitalograph; Maids Moreton, Buckingham, UK; and Spiropro; SensorMedics; Yorba Linda, CA), and four devices had acceptable limits of agreement for FEV1 (Datospir models 70 and 120, MicroLoop, and Pneumotrac). The Bland and Altman graphs of FVC, FEV1, and FEV1/FVC are submitted as on-line supplementary material.
|
|
|
Discussion
We report the results of a multicentric study of 10 office spirometers compared according to both their technical and user-friendliness characteristics. The precision of FEV1 measurement was good in the majority of these small spirometers and was comparable to the values obtained with the standard diagnostic devices. However, the repeatability of FVC was generally poorer, and the broad limits of agreement of FVC or FEV1 observed between some office spirometers and the standard devices may preclude the interchangeability of the results.
All the office spirometers presented in this study received the label "meets ATS recommendations." This label supposes that the spirometers were checked by a series of predetermined flow-volume curves via a computer-driven piston pump.12 In the standardization of spirometry published recently,16 the ATS/European Respiratory Society (ERS) task force recommends that spirometers should be evaluated using a computer-driven mechanical syringe or its equivalent. The use of the 24 ATS standard waveforms for FVC and FEV1 is recommended, but the recent ATS/ERS statement16 does not require that these tests must be performed by an independent laboratory, such as the laboratory at LDS Hospital for example. We requested the results of these tests from the manufacturers of the office spirometers. Datospir models 70 and 120, Simplicity, OneFlow, Pneumotrac, and SpiroStar (Medikro; Kuopio, Finland) devices were tested by their manufacturers using 24 ATS standard waveforms, and the other devices were tested at LDS Hospital. All the spirometers tested at LDS Hospital using the 24 standard ATS waveforms had fewer than three accuracy errors and repeatability errors for FVC or FEV1 at ambient testing conditions. The OneFlow met the ATS criteria for monitoring devices but not for diagnostic spirometers. At variance with previous ATS quality criteria for spirometers, there is no longer distinction between diagnostic and monitoring devices in the recent ATS/ERS statement.16 The same degree of accuracy is required for all devices, and corresponds to the former criteria (ATS) for diagnostic spirometers. Using the 24 ATS waveforms, the OneFlow had better accuracy results for FEV1 than for FVC, and only the FEV1 was within the criteria for diagnostic devices. Accordingly, in our study, the OneFlow performed better for FEV1 measurement than for FVC. By contrast, the results of the SpiroStar tested by its manufacturer (Medikro) showed no error for accuracy and intradevice testing with FEV1 and FVC, using the criteria for diagnostic devices, whereas this device presented large limits of agreement in our survey.
At variance with the 24 ATS waveforms, we did not use a fixed signal to test the devices, so the differences between in vitro and in vivo performances may be explained by the fact that different populations were studied. However, if we limit our analysis to the small sample (n = 9) of ETs who tested all the models, we can see that EasyOne presented unacceptable limits of precision for FVC. However, this device, tested by an independent laboratory (LDS Hospital), met the ATS (diagnostic devices) recommendations for accuracy and precision in measuring FVC and FEV1, like the Spirobank, whereas the latter obtained acceptable limits of precision in our study.
The tests done with the computer-controlled mechanical syringe indicate that the spirometers working under ideal laboratory conditions, at least at ambient conditions, are reliable, but they do not guarantee that the same performances will be obtained in real-life conditions, as patients are tested throughout the day, as it was the case in our study. Therefore, we and other groups17 propose that the spirometers should be tested both with waveform generators (on a bench) and with real patients.
The precision of FEV1 measured by the office spirometers was comparable to that of the standard spirometers. Only one device showed unacceptable limits of precision (> 200 mL) for FEV1 (± 0.295 L for the Simplicity).
All the office spirometers were less reproducible for the measurement of FVC when compared to the standard spirometers. FVC appears more difficult to measure by the small spirometers (Table 2). Three office spirometers measured FVC with a precision worse than 200 mL (Table 2).
Although statistically significant in some office spirometers, the biases observed for FEV1 or FVC are probably not clinically relevant. Even if we consider the accuracy criteria of the ATS for diagnostic devices12 as the limits of acceptable bias, all the office spirometers tested for FEV1 and for FVC met this criteria. However, the SpiroStar had a bias of – 4.9 for FEV1/FVC (Table 4). This device underestimated FVC and consequently overestimated FEV1/FVC ratio as compared to the diagnostic spirometers. Some stage 1 COPD patients could therefore be misclassified as stage 0 in that way.
The present data also show that the limits of agreements were generally larger for FVC than for FEV1. We fixed the acceptable limits of agreement by analogy with the short-term coefficients of repeatability of FEV1 (320 mL) and FVC (450 mL) measured in COPD patients.15 These coefficients corresponded to the maximum absolute limit of agreement times 0.9 as computed using the Bland and Altman analysis with our data. Hence, the limits we chose were 350 mL (FEV1) and 500 mL (FVC). Relatively wide limits of agreement were found with some office spirometers for FVC.
Underestimation of low FEV1 and overestimation of high FEV1 were also found with some office spirometers. This proportional difference may also result in an underestimation of the FEV1/FVC ratio and COPD misclassification.
We devised a new questionnaire about the user friendliness of office spirometers. The results of this small and informative survey suggest large similarities among the majority of the spirometers. Almost all of them have an interface with a personal computer, can be connected to a printer, and have correct help functions. An important issue is the automated quality assessment.1718 Most of the devices check the reproducibility and the acceptability of the spirometric maneuvers. Many spirometers display immediately messages about the quality of the "blow." The software offers mostly a large choice of reference tables and provides an automated interpretation. According to some authors,1920 this feature is not without danger of misinterpretation for uninformed users. Large differences were encountered for the facility to use during home visits. Some devices are readily excluded by their large size. Some spirometers display a flow-volume curve on a small screen on the device itself. The need for calibration of the spirometers is a controversial aspect.1718 It is not realistic to expect a GP to handle on a daily basis a 3-L calibration syringe and perform a calibration check. Most manufacturers claim that their device does not need any further calibration or even calibration checks. In fact, the majority of the office spirometers cannot be calibrated by the user, and calibration can only be checked. The new ATS/ERS statement16 on standardization of spirometry, published after the first submission of this study, requires that flow spirometer calibration be checked daily using different speeds of injection to verify that the spirometers measure accurately across a range of flows. More studies are needed to verify the stability of each device over months to years of use.
Another important property is the integration of the spirometric data into the patient data files. The management of the medical data is "core business" for the GP. However, technical specifications may vary largely from one country to the other. In the Belgian situation, the I-Med standard in XML (extensible markup language) format is becoming increasingly important as an interface between electronic medical data files and external information. Extensible markup language format is designed especially for Web documents and to enable the transmission, validation, and interpretation of data between applications and between organizations. There is plenty of room for improvement in international standardization in this domain, as it was proposed in the recent ATS/ERS guidelines for standardization of spirometry.16
We did not compare the quality and the cost of the mouthpieces, nor the facility of disinfecting the parts in contact with the exhaled air. We do not have any information on the long-term solidity of the devices, nor on their resistance to shocks during transportation, and to changes in humidity and temperature. We did not make any price/quality assessment. Last but not least, we have no feedback about the after-sales service of the different spirometer representatives. This could be of major importance in real-world circumstances.
Other limitations of the study should also be considered. Of course, all the office spirometers available on the market could not be tested during this study. However, we included office spirometers coming from Europe and the United States, showing different technical characteristics, and our sample was sufficiently diversified. It is noticeable that some of these devices (both hardware and software) are frequently upgraded or changed, so that the same brands sold now may differ from the devices tested. The devices tested were those available in Belgium in 2002.
Another issue is our assumption that standard diagnostic spirometers are accurate and precise. Indeed, if one standard diagnostic spirometer presents a proportional error, it will make the office spirometer look like it has a proportional error, even if this latter is accurate. In a multicentric study like the present one, the risk that all the reference spirometers are inaccurate is relatively low. Moreover, the office spirometers were compared to two types of laboratory spirometers: volumetric and hot wire. If only one reference spirometer presented a proportional error, the relation between the average and the difference of FEV1 would not be significant. The clinically acceptable limits for the bias, the precision, and the limits of agreement of FEV1 and FVC were fixed by analogy with the quality criteria of the ATS for diagnostic spirometers and with the coefficients of repeatability observed in COPD patients, because these limits are not available in the literature.
At variance with standard spirometers, the majority of the office spirometers were not calibrated during the study (Table 1). This may seen somewhat surprising in a laboratory study, but we wanted to respect the instructions of the vendors/manufacturers also in terms of calibration. We must keep in mind that these devices are devoted for the use by GPs and not for laboratory purposes.
The limits of precision of the office spirometers reported in this study refer to short-term repeatability. In real circumstances, the repeatability may decrease after several months if the users do not check regularly the spirometers, especially with screen-type pneumotachographs that commonly become clogged with secretions.
In conclusion, the availability of accurate, precise, and easy-to-use office spirometers is a condition for widespread use of spirometry in general practice. This survey shows that several office spirometers have a high score of user friendliness, but their instrumental properties are variable. All but one of the devices had excellent reproducibility for FEV1. Moreover, we observed low biases and acceptable limits of agreement for FEV1 in four office spirometers. However, several devices presented a proportional difference for FEV1, leading to underestimation or overestimation of results. Furthermore, the lack of precision of FVC and unacceptable limits of agreement between FEV1 and FVC may lead to a risk of misclassification of patients according to the GOLD criteria for COPD.
For all the office spirometers tested, more attention should be paid to improve the accuracy and precision of FVC because it limits the interchangeability of the spirometric data between the GP and the PFT laboratory. Import and export facilities of the spirometric data should also be improved. More investigation is needed to assess the overall quality of these rapidly changing instruments. Finally, we confirm that in vivo testing of the spirometers gives additional information to the strict in vitro benchmark study. This study could be used as model for the users and the manufacturers wishing to test new devices.
Appendix
The COPD Advisory Board is composed of Belgian pneumologists and GPs with the sponsorship of GlaxoSmithKline Belgium. Members of the COPD Advisory Group, Detection and Diagnostic Cell include Professor Yernault (deceased), Professor Louis, Professor Vincken, Professor Rodenstein, Professor Demedts, Dr. Gillard, Dr. Dierckx, Dr. Coolen, Dr. Robience, Mr. Schuermans, Dr. Cnockaert, and Mr. Rochette.
Acknowledgements
The authors thank the technicians of the pulmonary function laboratories: J. Onrubia, R. Vanleemputten, D. Schuermans, Y. Dewandeleer, F. Rochette, R. Juste, N. Celis, and S. Van Poyer. One of the ETs is a coauthor of this article (G.L.); the vendors: Mrs. Janssens, Mr. Du Four, Mr. Knez, Mr. Reyskens, Mr. Moens, Mr. Moreno, Mr. Servais, and Mr. Tastenoy; and our sponsor: GlaxoSmithKline Belgium (Mr. Dumoulin, Mr. Thierry, and Mr. Cleymans).
Footnotes
Abbreviations: ATS = American Thoracic Society; CI = confidence interval; ERS = European Respiratory Society; ET = expert technician; GOLD = Global Initiative for Chronic Obstructive Lung Disease; GP = general practitioner; PFT = pulmonary function test; Sw = within-subjects SD
This work was performed at Cliniques Universitaires Saint-Luc, Vrije Universiteit Brussel, and Katholieke Universiteit Leuven, Belgium.
Financial support was provided by GlaxoSmithKline Belgium, which hosted some of the investigator meetings.
The authors of this study have no financial interest in the subject of this article.
Received for publication July 26, 2005. Accepted for publication March 5, 2006.
References
This article has been cited by other articles:
|
|
 |
|
  M. J. Joo, T. A. Lee, and K. B. Weiss Geographic Variation of Spirometry Use in Newly Diagnosed COPD Chest, July 1, 2008; 134(1): 38 - 45. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
