|
|
||||||||
Guest Access | Sign In via User Name/Password |
|||||||||
* From the Imunoallergology Division (Drs. Fonseca, Castel-Branco, and Vaz), São João Hospital; and Biostatistics and Medical Informatics Department (Drs. Costa-Pereira, Silva, and Mr. Magalhães), and Immunology Department (Dr. Delgado), Faculdade de Medicina, Universidade do Porto, Porto, Portugal.
Correspondence to: Joao A. Fonseca, MD, Biostatistics and Medical Informatics Department, Faculdade de Medicina, Alameda Hernani Monteiro, 4202451 Porto, Portugal; e-mail: jfonseca{at}med.up.pt
| Abstract |
|---|
|
|
|---|
Design, setting, and participants: After detailed instructions, adults without airways diseases and patients with stable asthma attending an outpatient clinic performed four sets of expiratory maneuvers, one set for each device, in a strictly random order. Each set comprised three maneuvers with 2 to 3 min of rest between them.
Measurements: Reproducibility of FEV1 and peak expiratory flow (PEF) was assessed by a coefficient of variation (CV) and intraclass correlation coefficient (ICC), and accuracy was assessed by ICC and limits of agreement.
Results: Of the 38 participants evaluated, 71% were women and 61% had asthma. Ages ranged from 18 to 58 years, and FEV1 ranged from 1.2 to 4.8 L. In all monitoring devices, CV was < 6% and ICC was > 0.94 for the reproducibility of both FEV1 and PEF measurements. The accuracy of the PiKo-1 device was better for FEV1 (ICC = 0.98) than for PEF (ICC = 0.90). The Spirotel device had similar results for FEV1 and PEF (ICC = 0.95). The Mini-Wright device had the lowest accuracy (ICC = 0.87), particularly for PEF values < 500 L/min.
Conclusions: These low-cost and easy-to-use electronic monitoring devices showed a very good reproducibility and were in agreement with the pneumotachograph. Therefore, the PiKo-1 and Spirotel devices seem adequate for both screening and monitoring. However, prospective studies are still needed to assess their long-term reproducibility and usability and, particularly, the effects on the improvement of respiratory care.
Key Words: accuracy agreement asthma electronic health technology assessment peak flowmeter pulmonary function reproducibility
| Introduction |
|---|
|
|
|---|
Peak expiratory flow (PEF) monitoring has been strongly recommended by asthma guidelines,2 and clinical studies34 have shown that the routine use of peak flow meters (PFMs), along with a self-management plan and education program, can lead to better control of asthma. In the last decades, mechanical PFMs have been the most used method of lung function monitoring. They provide a simple, quantitative, and reproducible measurement of large airways function. Nevertheless, several difficulties have impaired their wide use, and some authors567 have questioned the need for home monitoring of PEF. The reasons put forward were the low compliance by the patients and the limitations of traditional PFM that have both low accuracy and insensitivity to changes.89 These devices have other limitations, including errors in data produced by the patient, related with exclusive use of PEF, which is highly dependent on the patient effort and errors in data recording, mostly transcription errors but also forged registries.1011
These limitations may be overcome with electronic monitoring devices if they prove to be reproducible and accurate, as they are already capable of recording and transmitting data for clinical analysis. It would also be beneficial to monitor additional parameters along with PEF.1213
A growing number of low-cost devices have been recently developed. This has special importance to developing countries, where respiratory disease is a major public health problem and pulmonary function measurements are largely unavailable.
Most of the pulmonary function monitoring devices comply with American Thoracic Society (ATS) recommendations when tested using simulated standard waveforms by a computer-driven mechanical syringe. The performance of these devices in humans and in clinical settings is largely unknown. It is not known how they compare to the current methods used in the clinic.14
Agreement studies1516 compare the performance of an instrument throughout repeated measurements (reproducibility) and also with a reference (accuracy or validity). Reproducibility assessment is the first step in agreement studies; if acceptable, the accuracy should be checked before use in clinical trials or other clinical settings.
This study aims to compare in stable asthma patients and individuals with normal airways the following: (1) the within-session reproducibility of two pulmonary function electronic monitoring devices (PiKo-1; Ferraris Respiratory Europe; Hereford, UK; and Spirotel; MIR; Rome, Italy) with the widely used Mini-Wright Peak-Flow Meter (Clement-Clarke International; Harlow, Essex, UK) during the same set of maneuvers; and (2) to evaluate their accuracy comparing with an office pneumotachograph as reference. Our hypothesis is that the measurements of PEF and FEV1 of the two electronic monitoring devices are in close agreement with those of a pneumotachograph. We also hypothesized that those measurements have better discriminative properties than the measurements of one of the most used PFMs in Europe: the Mini-Wright PFM.
| Materials and Methods |
|---|
|
|
|---|
25. The PiKo device has an optional, serial interface cradle to allow downloading PiKo-1 data to a computer and companion software, allowing communication of results using the Internet to track and trend patient data. This optional cradle and personal software costs
25 more.
The Spirotel device is a turbine with an infrared interruption spirometer and has a built-in modem and an optional oximeter. It was developed both for screening in the doctors office and for home-care monitoring. The Spirotel device records spirometry parameters including FVC; FEV1; percentage of predicted FEV1; PEF; forced expiratory flow, midexpiratory phase; forced expiratory time; flow/volume curve; and date and time of the test. It can also record symptoms and the responses to programmable questions. Its has a flow range of ± 16 L/s and a maximal volume of 10 L, a flow accuracy of 5% or 200 mL/s, and a volume accuracy of 3% or 50 mL, whichever is greater. Each unit costs approximately
400. Both PiKo-1 and Spirotel devices have been laboratory tested (data on file), and both met or exceeded the latest ATS accuracy standards.
The standard range version of the Mini-Wright PFM was used. It has 10-L increments from 60 to 800 L/min, and its performance has been studied.17181920212223 Each unit costs approximately
20. The monitoring devices used in the study were new and were acquired directly from the manufacturers without their knowledge of our purpose.
A widely available, well-known technology was used as reference: a Fleisch-type pneumotachograph (model 2120; Vitalograph; Maids Moreton, Buckingham, UK).24 Each day, the pneumotachograph was calibrated using a 3-L syringe. All other devices were calibrated before the study and, in accordance to the manufacturers, did not required calibration during the time period of the study.
Population
Patients attending an asthma and allergy outpatient clinic of a teaching hospital between 10 AM and 12 noon of 20 alternate days during a period of 12 weeks were invited to participate in the study. Patients were considered eligible for participation if they were > 17 years old, had a documented medical diagnosis of asthma, were currently receiving prescribed medication for asthma, and were clinically stable. Stability was defined as no asthma exacerbation or acute illness in the last 4 weeks, and no clinical indication of deterioration of asthma control in the last week. No pulmonary function exclusion criteria were established. The diagnoses of airways diseases other than asthma or neuromuscular or psychiatric diseases were exclusion criteria. Two groups of participants were defined: asthma patients (asthmatics) and patients without any airways disease (normal subjects). Asthmatics had a previous medical diagnosis of asthma, were currently receiving asthma medications, and were attending the clinic for asthma follow-up. Normal subjects were defined as patients followed up at the clinic for allergic diseases without airways involvement and with no history of pulmonary disease.
Study Protocol
In Figure 1
, a schematic representation of the study is presented. After providing written informed consent, patient demographics, height, weight, smoking status, previous medical diagnosis, current medical status (including acute illnesses in previous 4 weeks), and inhaled medication in previous 12 h were assessed. To ensure clinical stability, patients completed the Asthma Control Questionnaire25 and a modified Borg dyspnea scale that was repeated at the end of the expiratory maneuvers.26 The self-administered version of Asthma Control Questionnaire has six questions regarding asthma control in the previous week; scores range from 0 to 6 (no control). The modified Borg dyspnea scale has a range from 0 (no dyspnea) to 10 (maximal dyspnea).
|
Statistical Analysis
Within-session reproducibility was defined as the agreement of the measurements performed with the same device and individual during one set of maneuvers. Within-session reproducibility was assessed between the two best maneuvers by the coefficient of variation (CV) and the intraclass correlation coefficient (ICC). The association between the PEF and FEV1 measurements by the pneumotachograph and the different monitoring devices were plotted with the respective regression lines. Accuracy was defined as the agreement of measurements performed with measurements performed with a reference device in the same individual.
Considering the pneumotachograph as the reference instrument, the accuracy for PEF and FEV1 measurements was assessed by the determination of the ICC with the monitoring devices measurements as dependent variables and by the limits of agreement according to Bland and Altman.28 The mean differences between each electronic monitoring device and the pneumotachograph data were plotted against the mean values of FEV1 and PEF from each device and the pneumotachograph, and limits of agreement were estimated at ± 2 SD of the differences. The random error, computed as 1-r2, was defined as the deviation of the tested device values from the regression line. This random error, sometimes named precision, was considered another proxy for accuracy.
Statistical analysis was carried out using statistical software (SPSS version 11.5; SPSS). A probability of < 5% was considered to be significant. For ICC and CV, 95% confidence intervals (CIs) were calculated.
| Results |
|---|
|
|
|---|
|
5% for both PEF and FEV1 (Table 2
).
|
|
|
|
| Discussion |
|---|
|
|
|---|
The inadequate performance of the Mini-Wright PFM is consistent with the observations of other studies.17181922 In fact, a model with a different scale has been introduced in the market to reduce errors.29 The lack of linear response is not necessarily characteristic mechanical PFMs.30
To our best knowledge, no studies have been published regarding the clinical performance of the PiKo-1 or Spirotel devices. Other home spirometers and electronic monitoring devices have been studied, indicating their performance is fairly adequate for different clinical situations.3132333435363738
Limitations and Strengths
Our participants had a relatively high pulmonary function, which is probably related to the clinical stability inclusion criterion. This criterion was set in order to ensure the security of the protocol and to avoid maneuver-induced bronchospasm.39 These objectives have been achieved as indicated by the observation of measurement values (data not shown) and the minimal change in the dyspnea scale after the maneuvers. Nevertheless, the population studied is quite typical of the asthma patients seen in most medical settings, as the majority of asthma severity is mild, and also patients with more severe asthma have normal or near-normal pulmonary function after adequate treatment. In fact, clinical evaluation of medical devices and diagnostic tests should be performed with subjects with similar characteristics to those who are expected to benefit from their use.40 Some authors3239 also recommend that the assessment include a second group of subjects without the disease under study; we did this by including 15 subjects with no respiratory disease.
In our study, only 42% of subjects had previously performed spirometry; this can be regarded as strength of this study because if we had used experienced, too-well-trained subjects, this would have biased positively the results. We chose to use a pneumotachograph as reference. This is an accepted way of testing PFMs.20 In fact, in clinical settings the use of monitoring devices along with office spirometers is much more frequent than with sophisticated laboratory equipment.
We have not fully evaluated the measurement capabilities of the Spirotel device; parameters such as FVC or forced expiratory flow, midexpiratory phase were not focused on in this study. Also, we did not intend to study the recording and transmitting functions of the devices. Finally, this study did not assess the reproducibility over a period of time. Long-term reproducibility may be more important in monitoring than accuracy if the devices are to be used to assess variations of the disease over time.
Implications for Clinical Practice
The availability of a simple, low-cost device with a very good agreement with a pneumotachograph may have a profound impact in low-income countries where pulmonary function laboratories cannot be easily implemented. In spite of the good agreement between monitoring devices and the pneumotachograph, the devices are not interchangeable as can be inferred by the outliers, albeit in small number, observed in Figure 3. When treating a patient, a measurement with one device should not be compared directly with a measurement done with another device, a well-known warning for other pulmonary function devices.
It is expected that the storage of data by electronic monitoring devices diminishes transcription errors and allows a better assessment of compliance. Also, the possibility of measuring other parameters such as FEV1 along with PEF may be advantageous in long-term management of respiratory disease. Of the monitoring devices tested, only the Spirotel device can measure FVC and FEV1/FVC, which are also relevant in many clinical situations. Other particular features of Spirotel device, such as the built-in modem and optional oximeter, can also prove interesting for specific settings.
The cost of the PiKo-1 device is much less than most other pulmonary function electronic monitoring devices and is similar to a mechanical PFM. In light of the results of this and other studies,1123 the acquisition of mechanical PFM may not be the best option.
Future Research
Future work is needed in order to determine the clinical performance of the PiKo-1 and Spirotel devices in children, in COPD patients, and in unstable asthma patients with lower pulmonary function. Furthermore, it is necessary to study their long-term reproducibility, assess practical utilization issues (such as battery life and download of data), other parameters measured (in the case of the Spirotel device), and the acceptability by the patients of electronic monitoring. Finally, the impact of electronic monitoring devices in improvement of asthma and COPD clinical and economic outcomes also needs to be established.
| Conclusion |
|---|
|
|
|---|
| Acknowledgements |
|---|
| Footnotes |
|---|
Participating institutions (Hospital São, Joao and Faculdade de Medicina da Universidade do Porto) provided all the financial support for this study.
Received for publication December 24, 2004. Accepted for publication February 17, 2005.
| References |
|---|
|
|
|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |