| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
Guest Access | Sign In via User Name/Password
|
|
|
|||||||
Guest Access | Sign In via User Name/Password |
|||||||||
* From LDS Hospital and University of Utah (Drs. Jensen and Crapo, and Ms. Howell), Salt Lake City, UT; and Pfizer Global Research and Development (Drs. Teeter, England, White, and Pickering), Groton, CT.
Correspondence to: Robert L. Jensen, PhD, Pulmonary Laboratory, LDS Hospital and University of Utah, Eighth Ave and C St, Salt Lake City, UT 84143; e-mail: ldrjens1{at}ihc.com
Abstract
Background: The objective of the study was to characterize the biological and technical components of variability associated with longitudinal measurements of FEV1 and carbon monoxide diffusing capacity (DLCO). Variability was apportioned to subject and instrument for five commercially available pulmonary function testing (PFT) systems: Collins CPL (Ferraris Respiratory; Louisville, CO); Morgan Transflow Test PFT System (Morgan Scientific; Haverhill, MA); SensorMedics Vmax 22D (VIASYS Healthcare; Yorba Linda, CA); Jaeger USA Masterscreen Diffusion TP (VIASYS Healthcare; Yorba Linda, CA); and Medical Graphics Profiler DX System (Medical Graphics Corporation; St. Paul, MN).
Methods: This was a randomized, replicated cross-over, single-center methodology study in 11 healthy subjects aged 20 to 65 years. Spirometry and DLCO measurements were performed at baseline, 3 months, and 6 months. Repetitive simulations of FEV1 and DLCO were performed on the same instruments on four occasions over a 90-day period using a spirometry waveform generator and a DLCO simulator.
Results: The coefficient of variation associated with repetitive measurements of FEV1 or DLCO in subjects was consistently larger than that associated with repetitive simulated waveforms across the five instruments. Instrumentation accounted for 13 to 58% of the total FEV1 and 36 to 70% of the total DLCO variability observed in subjects. Sample size estimates of hypothetical studies designed to detect treatment group differences of 0.050 L in FEV1 and 0.5 mL/min/mm Hg in DLCO varied as much as four times depending on the instrument utilized.
Conclusions: These results provide a semiquantitative assessment of the biological and technical components of PFT variability in a highly standardized setting. They illustrate how instrument choice and test variability can impact sample size determinations in clinical studies that use FEV1 and DLCO as end points.
Key Words: clinical trial design • diffusing capacity • diffusing capacity simulator • pulmonary function testing • pulmonary waveform generator • spirometry • variability
The sources of variation in lung function measurement have been described by Becklake and White,1 but the relative contributions of technical (instrument) and biological elements have not been well studied, especially for carbon monoxide diffusing capacity (DLCO) and lung volumes. Much of the early focus was on the technical aspects, specifically improving spirometers. The development of standard spirometric waveforms and mechanical simulators that could deliver them accurately allowed reliable testing of spirometer performance and has been included in current American Thoracic Society (ATS) and European Respiratory Society spirometry standards.23 In 1990, Nelson et al4 studied 62 different spirometer models using a mechanical spirometry waveform simulator and the ATS standard waveforms. Of the spirometers tested, 29% failed the tests and 14.5% were judged marginal. In all cases, specific problems were identified by the testing; when those problems were corrected, the spirometers passed.
Two of the authors (R.C. and R.J.) have since developed a DLCO simulator to eliminate biological qsources of variability,5 thereby making it possible to better identify instrument errors. The DLCO simulator utilizes two precision syringes in conjunction with a high precision mix of gases (tracer gas and carbon monoxide) to simulate a range of physiologically relevant DLCO values. Although there is no mandate for its use, some manufacturers have adopted it as part of their quality control.
This study was designed to quantify biological variability in spirometry and DLCO measurements over a 6-month period using a variety of modern instruments. The impetus for the study was the need to determine sample size and measurement frequency for clinical studies.
Materials and Methods
This was a randomized, replicated, cross-over, single-center methodology study to assess intrainstrument and intrasubject variability of pulmonary function testing (PFT) measurements over a 6-month period. The local institutional review board approved the study protocol, and the study was conducted in accordance with the ethical principles of the Declaration of Helsinki. All subjects gave written informed consent.
Instruments Tested
One new instrument model from each of five PFT equipment manufacturers was purchased. The instrument manufacturers and models were as follows: Collins CPL (Ferraris Respiratory; Louisville, CO); Morgan Transflow Test PFT System (Morgan Scientific; Haverhill, MA); SensorMedics Vmax 22D (VIASYS Healthcare; Yorba Linda, CA); Jaeger USA Masterscreen Diffusion TP (VIASYS Healthcare; Yorba Linda, CA); and Medical Graphics Profiler DX System (Medical Graphics Corporation; St. Paul, MN). These instruments were purchased by the study sponsor, Pfizer Inc., for the purpose of validating instruments being considered to measure PFT end points in clinical drug trials. Each instrument was set up and maintained according to manufacturer specifications. All PFT instruments were powered on continuously for the duration of the study. Instruments were calibrated or calibration checked according to manufacturer specifications on test days.
Study Population
Nonsmoking male and female subjects (20 to 65 years old) with no history of respiratory diseases or symptoms who passed a screening history and physical examination were eligible for inclusion. Women who were pregnant, lactating, or not using adequate contraception were excluded. Additional exclusion criteria were as follows: history of smoking in the 2 years prior to screening or a life-long total of > 5 pack-years; history of recreational drug use in the year prior to the study; recent eye surgery; treatment with any asthma medications or corticosteroids (except nasal corticosteroids for allergic rhinitis); any respiratory tract ailment in the 6 weeks prior to the study; inability to perform acceptable quality PFT at screening; recent blood donation; or any health condition that would, in the judgement of the investigators, interfere with the study. Lung function exclusion criteria were as follows: FVC or FEV1 > 120% or < 70% of predicted,6 or FEV1/FVC < 70%, or DLCO >120% or < 70% of predicted.67 Upper and lower exclusion limits were used to minimize regression to the mean.
Assessments
At the initial screening visit, the following were obtained: a medical history and physical examination including measurements of height and weight, and measurements of spirometry and DLCO using clinical laboratory instruments. Subjects were required to have negative alcohol breathalyzer (AlcoMate CA 2000 Digital Alcohol Detector; Wookyung Tech; Incheon City, South Korea) results at screening to continue study participation.8 Blood samples for hemoglobin concentrations were obtained at screening and weeks 12 and 24 and used to adjust DLCO values to a standard hemoglobin concentration according to ATS recommendations.9 Room temperature and barometric pressure were recorded on test days. Adverse events were collected at each study visit.
PFT and spirometry were performed on each subject at three time points: baseline (0 to 2 weeks), 3 months (12 to 14 weeks), and 6 months (24 to 26 weeks). All PFT was performed at the LDS hospital (Salt Lake City, UT) by two experienced technicians according to ATS standards.910 DLCO washout and sample volumes were fixed on the Collins CPL and were not adjustable in the Morgan, Jaeger, or Medical Graphics devices. Only on the SensorMedics Vmax were DLCO alveolar samples adjusted when thought necessary in the judgement of the technician performing the test. A computer-generated randomization scheme was utilized to ensure that each subject was tested on each instrument in a unique sequence. At each testing interval, subjects were tested on each instrument twice in a 2-week period with a restriction that they could only be tested on one instrument per day. On each test day, subjects were required to complete three acceptable FVC and three acceptable DLCO maneuvers on the selected instrument. The largest measured FEV1 from the three acceptable trials was recorded as the FEV1 value for that test day. All three acceptable DLCO measurements were recorded for each test day.
High-precision gas mixtures (± 1% accuracy) were used. Gases provided by the manufacturers have accuracies of up to ± 3%. Precision gas mixtures were provided by Hans Rudolph, Inc. (Kansas City, MO). They consisted of carbon monoxide, a tracer gas, 25% oxygen, and balance nitrogen. The test gases for each of the instruments used were as follows: SensorMedics Vmax and Collins CPL, 0.3% CO and 0.3% CH4; Jaeger and Morgan, 0.3% CO and 10% He; and Medical Graphics, 0.3% CO and 0.5% Ne. Twenty-five percent oxygen was used in the precision gas mixtures to establish an alveolar pressure of oxygen in test subjects close to sea level (Salt Lake City elevation 1,400 m).11 A total of six DLCO tests in three subjects were tested with gas mixtures containing 21% oxygen. The DLCO values obtained were subsequently adjusted to reflect the difference in oxygen concentrations using the method described by Kanner and Crapo.12 Gas concentrations were gravimetrically determined to be within ± 1% absolute.
Simulator Testing
In a separate study reported in a companion article,13 each of the five PFT instruments used in the present study also underwent simulations using a pulmonary waveform generator and DLCO simulator. Data from the simulations, in combination with results from the present study, were used to derive the components of variability attributable to instrument and biological sources.
Statistical Analysis
Data from all available time points were used to determine subject- and simulator-derived group mean and range values for FEV1 and DLCO for each instrument tested. The variability associated with repeated simulated or human subject testing was determined from the root mean square coefficient of variation (RMSCV), calculated as the square root of the average squared coefficient of variation (CV) [CV = SD/mean] for all simulations or subjects across all time points. The ratio of instrument to subject RMSCVs was used to assess the percent of total variability due to instrumentation. Sample size estimate curves for hypothetical studies assessing treatment group differences in change from baseline FEV1 or DLCO and assuming a power of 80% and an 
value of 0.05 were generated for each instrument tested utilizing the RMSCV observed in subject testing (NQuery Advisor 5.0; Statistical Solutions; Saugus, MA).
Results
All 11 subjects screened were assigned to the study (Table 1 ). Eight subjects completed all study visits. One subject was discontinued on day 17, and another subject was discontinued on day 100; both withdrew consent. An additional subject was discontinued on day 100 due to pregnancy. All PFT data collected were included in the analysis except for the subject discontinued at day 17 and three DLCO measurements from weeks 24 to 26 on the Morgan instrument. These data were excluded because of the discovery of a small leak in the Morgan instrument that could affect DLCO; the leak was subsequently corrected.
|
|
|
|
|
|
|
Discussion
To our knowledge, this is the first study to compare biological and instrument sources of variation among modern, commercially available PFT instruments. Using a combination of both human and simulator testing, we estimated the magnitude of instrument and biological sources of variation in lung function test results. Several important elements were described: (1) the observed DLCO variability in subjects was largely due to instrument variability (35 to 68%); (2) DLCO variability varied by a factor of approximately 2 across the models tested; and (3) as expected, overall FEV1 variability was approximately half the DLCO variability, and the instrument variability associated with FEV1 simulator testing was smaller than DLCO instrument variability.
The ranges of FEV1 and DLCO were relatively wide for both human subject and simulator testing. The study was specifically designed to span a wide range of simulator DLCO and spirometry values, representing values likely to be seen in clinical testing. The human subject volunteers provided a wide range of values for both FEV1 (approximately 2.2 to 5.6 L) and DLCO (approximately 19 to 41 mL/min/mm Hg [6.36 to 13.73 mmol/min/kPa]). The study subjects were healthy, and therefore our results may not apply to patients with clinically abnormal test values. Variability would likely be larger if patients with disease had been included, although a study14 measuring short-term variability in FEV1 in patients with obstructive lung disease found no significant difference from normal healthy subjects.
The biological variability in this study can be explained by several interacting elements. Many of the known elements contributing to biological variability (ie, technician-subject interactions, circadium rhythms, transient illness, underlying lung disease, alcohol ingestion, or fluctuations in serum hemoglobin values)234 should have been minimized by the procedures utilized in our study. It seems likely, therefore, that a significant component of the variability not due to instrumentation is reflective of the unique procedures imposed by each device and the physical interaction of the subjects with the individual instruments tested. The specific elements of these subject-instrument sources of biological variability remain the subject of future investigations.
In general, there appeared to be a direct correlation between the degree of variability associated with the measurement of FEV1 and DLCO identified through simulator and human subject testing (Fig 1, 2). An exception to this was the measurement of FEV1 on the Jaeger instrument, which demonstrated the highest simulator yet lowest degree of human subject variability (Fig 1). As demonstrated in the companion article to the present study, the Jaeger instrument produced ATS-defined errors10 in the measurement of FEV1 (> ±3.5% or ± 0.070 L of simulator target) in 62 of 480 instances.13 These errors occurred in the measurement of waveforms 1, 3, 4, 5, 6, 8, 11, 12, 13, 17, 19, 20, 22, 23, and 24, most of which simulate abnormal volume-time profiles. In contrast, all of the subjects tested in the present study had normal lung function. In the present study, we did not directly assess the specific causes of the variability identified. However, we speculate that apparent lack of correlation between instrument and patient variability demonstrated on the Jaeger instrument tested could have been related to its inability to accurately measure the FEV1 of abnormal ATS waveforms.
Using the estimates of variability from the human subject testing for each device, we derived the group sample sizes necessary to detect significant differences in FEV1 and DLCO. These analyses showed that for FEV1, two of the five instruments tested required sample sizes almost twice as large as the remaining three devices. For example, to detect a relative difference in FEV1 of 0.05 L, two of the devices required a sample size of approximately 125, whereas the remaining three required a sample size of approximately 75 (Fig 3). The results for DLCO were markedly different. All five instruments behaved differently, and the necessary sample sizes ranged from just > 100 to > 500 to detect a DLCO difference of 0.5 mL/min/mm Hg (0.17 mmol/min/kPa) between study groups (Fig 4). This represents a fivefold increase in sample size using one device compared to another. Differences in sample size estimates are reflected in direct costs to perform a study and increasing complexity in deploying clinical trials. Some clinical trials will not be attempted if estimated sample sizes are too large.
The instruments tested in this study are representative of models commonly used in clinical PFT laboratories. The Collins CPL was the standard instrument utilized in a multicenter clinical trial examining the pulmonary safety of inhaled human insulin (recombinant DNA origin) [Exubera Inhalation Powder; Pfizer; New York, NY], recently approved in the United States and European Union for the treatment of adult patients with type 1 or type 2 diabetes mellitus.15 In that study, 226 subjects completed a total of 4,057 DLCO test sessions at 33 clinical sites over a 24-week period. The mean intrasubject CV observed for these tests was 5.85%. This degree of DLCO variability was only slightly more than that achieved in subjects on the Collins unit in this study, and was actually lower than three of the other instruments tested in the present study (Table 3). This suggests that the DLCO variability observed in human subjects in our study can be achieved in multicenter trials using standardized instruments and methodology.
Our study has several limitations. Only one device from each manufacturer was tested. This limits the general conclusions we can draw about any manufacturer or device model. In addition, it was essentially a best-case scenario. Highly experienced technicians performed the testing, and each instrument was scrupulously calibrated and maintained according to manufacturer recommendations and serviced if any malfunction occurred. Testing was carefully performed according to ATS standards and all manufacturer recommendations. Simulator and human testing were performed under highly controlled conditions at a single site. Real-world variability may therefore be greater than this study estimates.
Two aspects of human testing are not fully "simulated" with the DLCO simulator. The first is the exact gas conditions (temperature and humidity) that would normally be seen in human testing. Second is the more gradual change in gas concentrations in humans that occurs during the exhalation phase of the DLCO maneuver than is introduced with the DLCO simulator. The first aspect is compensated by the humidity correction and reduction or drying systems of each device (drying tubes, CO2 scrubbers, and H2O absorbers), and the second aspect is controlled because experienced technicians appropriately set all sample and washout volumes in the devices when adjustments could be made.
In summary, examining human subject variability and combining this with data from an independent study of simulated testing assessing device variability have allowed us to quantitatively partition the biological and technical components of the variability for longitudinal measurements of FEV1 and DLCO. Our results illustrate how instrument choice and test variability can impact sample size determinations and thus have direct implications for the design of clinical trials that utilize FEV1 and DLCO as end points.
Acknowledgements
The authors wish to thank Janet Embry for manuscript review and editing and Angie Flint for technical support and data collection. Editorial support was also provided by J. Grice of PAREXEL and was funded by Pfizer Inc.
Footnotes
Abbreviations: ATS = American Thoracic Society; CV = coefficient of variation; DLCO = carbon monoxide diffusing capacity; PFT = pulmonary function testing; RMSCV = root mean square coefficient of variation
The study was sponsored by Pfizer Inc.
Drs. Jensen and Crapo are consultants for Pfizer Inc, and both received royalties from Hans Rudolph Company. Drs. Teeter, England, and Pickering, and Ms. White are employed by Pfizer Inc. Ms. Howell has no conflict of interest to disclose.
Received for publication August 11, 2006. Accepted for publication February 16, 2007.
References
Related Article
Related Editorial
This article has been cited by other articles:
|
|
 |
|
  N. MacIntyre Finding Signals Amidst the Noise in Pulmonary Function Testing Chest, August 1, 2007; 132(2): 367 - 368. [Full Text] [PDF]
|
|
|
|
|
|
R. L. Jensen, J. G. Teeter, R. D. England, H. J. White, E. H. Pickering, and R. O. Crapo Instrument Accuracy and Reproducibility in Measurements of Pulmonary Function Chest, August 1, 2007; 132(2): 388 - 395. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
