Chest ACCP Education Calendar
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
QUICK SEARCH:   [advanced]


     

Guest Access | Sign In via User Name/Password
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF) Free
Right arrow Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Article Archive
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via ISI Web of Science (3)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Pérez-Padilla, R.
Right arrow Articles by Olaiz, G.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Pérez-Padilla, R.
Right arrow Articles by Olaiz, G.
(Chest. 2003;123:1090-1095.)
© 2003 American College of Chest Physicians

Spirometric Variability in a Longitudinal Study of School-Age Children*

Rogelio Pérez-Padilla, MD; Justino Regalado-Pineda, MD; Laura Mendoza, MD; Rosalba Rojas, MD; Víctor Torres, MD; Víctor Borja-Aburto, MD and Gustavo Olaiz, MD; on behalf of the EMPECE Study Group{dagger}

* From the Mexican National Institute of Respiratory Diseases (Drs. Pérez-Padilla and Regalado-Pineda), the National Institute of Public Health (Drs. Mendoza and Rojas), and the Environmental Health Directorate of the Secretariat of Health (Drs. Torres, Borja-Aburto, and Olaiz), Mexico City, Mexico. {dagger} Other members of the EMPECE study group are given in the Appendix.

Correspondence to: Rogelio Pérez-Padilla, MD, Instituto Nacional de Enfermedades Respiratorias, Tlalpan 4502, México DF, México; e-mail: perezpad{at}servidor.unam.mx


   Abstract
TOP
 Abstract
Introduction
 Materials and Methods
 Results
 Discussion
 Appendix
 References
 
Objectives: To describe spirometric reproducibility in a longitudinal study of students from Mexico City, and also the frequency of subjects fulfilling quality criteria proposed for children.

Subjects and methods: Three thousand three hundred forty-seven participants from the third through sixth grades of elementary school were recruited to perform biannual spirometry, yielding a maximum of seven evaluations and a total of 15,563 tests. Standard recommendations of the American Thoracic Society (ATS) were followed, using dry rolling-seal volume spirometers.

Results: During their first spirometric test, > 95% of the subjects fulfilled each of the quality criteria proposed by ATS for adults, though not all of them did so simultaneously. For example, only 72.4% obtained three acceptable maneuvers, reproducibility for FEV1 and FVC to < 200 mL, and a small back-extrapolated volume that increased to 92.3% by the second test. Between phase 1 and phase 7 of the study, spirometry quality increased significantly, as a result of subject and technician training. Intratest and intertest (with a 6-month difference) spirometric variability was less in boys than in girls. Intratest variability was also lower in younger and taller subjects. Technicians contributed significantly to intratest and intertest variability, the latter decreasing if the same technician performed both evaluations.

Conclusion: Children > 7 years old can fulfill ATS criteria of quality after the first spirometric evaluation. To maintain quality of spirometric tests in longitudinal studies of children, a strict control is required, especially of technician performance.

Key Words: American Thoracic Society criteria • children • European Respiratory Society criteria • quality control • spirometry


   Introduction
TOP
 Abstract
 Introduction
Materials and Methods
 Results
 Discussion
 Appendix
 References
 
In 1996, a longitudinal study in Mexico City was undertaken with the main objective of studying the health effects of air pollution on primary school students. In all such studies, quality control is important to increase the signal-to-noise ratio. Components of quality control programs in spirometry have been described in detail1 but mostly for adults. Few studies have analyzed the spirometry performance of children in long-term studies. The objective was to describe the variability of spirometry in school-age children and to analyze the applicability of international standards of spirometry to children.


   Materials and Methods
TOP
 Abstract
 Introduction
 Materials and Methods
Results
 Discussion
 Appendix
 References
 
Students were selected from schools located < 2 km from 10 air-pollution monitors that form part of an automated web of environmental monitoring equipment in metropolitan Mexico City. All zones of Mexico City and all types of schools—public, private, daytime, and afternoon—were studied. Thirty-five schools were selected by simple random sampling, and all children between 8 years and 12 years of age attending third to sixth grade were invited to participate. The students were required to provide written consent from their parents or tutors in order to participate. The protocol was approved by the Ethics Committee of the National Institute of Respiratory Diseases, and also by the Secretariat of Public Education and authorities of Mexico City and the State of Mexico. All participants received their written spirometric results, and those with decreased functions were recommended to consult a physician and were given the option of an assessment by the National Institute of Respiratory Diseases. New students arriving at the selected classes were accepted into the study, except those who requested entry during the final evaluation. Students who changed schools during the study could only be followed up if admitted to another participating school.

Methodology for Spirometric Examination
Tests were done with six identical dry rolling-seal horizontal volume spirometers (SensorMedics Model 922; SensorMedics; Yorba Linda, CA). These spirometers fulfill the quality criteria recommended by the American Thoracic Society (ATS).1 2 Routine calibration was performed with a 3-L syringe (SensorMedics), and quality control checks were performed with a flow-volume syringe (Flow-Volume Calibrator; Jones Medical Instruments; Oak Brook, IL).

Before each study, height and weight without shoes were measured. Spirometer temperature was measured by a digital wall thermometer and included in the data along with the mean barometric pressure of Mexico City (583 mm Hg).

Technicians were trained specifically for this project, using the guidelines of the Spirometry Course of the National Institute of Occupational Safety and Health. Four full-time field supervisors participated, two of them in charge of spirometry quality control, along with 12 technicians. Standard procedures suggested by ATS1 were followed, with additions coming from a similar study3 that involved students of the same age.

To avoid inhaling from spirometers, only the expiratory part of forced expiratory maneuvers was registered. A maximum of nine forced expiratory maneuvers were performed to complete three acceptable readings according to ATS criteria; each was identified automatically by a nil error code. That is, each maneuver identified with a nil error code has a back-extrapolated volume (Vbe) <= 5% of FVC or <= 100 mL, an expiratory time of >= 6 s, or a zero volume change (< 50 mL) for the last 2 s of expiration. In addition, reproducibility for FVC and FEV1 within 5% was identified automatically.

The three best maneuvers were saved by the computer. In general, < 5 maneuvers (done while standing) were required to obtain three acceptable readings. If the child had a tendency to inspire during the FVC maneuver and continue expiration later, the maneuver was repeated using nasal clips.

At the end of each study, technicians reviewed a quality control screen on the monitor to certify the correct selection of best maneuvers and values by the machine. Spirometric technique was supervised at least once a week by the quality control staff. To analyze only reliable data, at the end of a season all spirometric tests carried out were reviewed once again.

The quality of spirometric tests was assessed by several criteria in addition to the automatic evaluation done by the software. One was the number of acceptable maneuvers according to ATS,1 ranging from 0 to 3, the highest kept by the spirometric software. Another indicator of quality was reproducibility. FEV1 and FVC were considered reproducible according to ATS criteria when the best two trials differed by not more than 200 mL. In addition, intratest reproducibility was quantified by the difference between the two best measurements of FEV1, FVC, forced mid-expiratory flow rate (FEF25–75), and peak expiratory flow rate (PEFR) in original units and expressed as percentages (percentage difference between the two highest FEV1 values [dFEV1%], percentage difference between the two highest FVC values [dFVC%], percentage difference between the two highest FEF25–75 values [dFEF25–75%], and percentage difference between the two highest PEFR values [dPEFR%]) and the coefficient of variation (COV)4 of the same variables.

Quality criteria were also calculated as described by Enright et al5 in the Lung Health Study. These criteria include, in addition to dFEV1% and dFVC%, the reproducibility of PEFR (absolute difference between the two highest PEFR values [dPEFR]), which is much more difficult to achieve. In addition, performance for flows and volume is estimated separately. The original classification was used for flow quality, but for volume it was modified slightly because spirometers did not provide the expiratory time. For the purposes of this study, volume "A" grade was assigned if the average time to reach vital capacity (FET100%VC) in the three best maneuvers was > 6 s. FET100%VC is shorter than expiratory time if vital capacity can be reached before expiratory effort ends; in other words, when a volume plateau is present, a common event in children.

As indicators of intrasubject, intertest reproducibility (6-month difference), the Pearson correlation coefficient,6 the intraclass correlation coefficient, and the COV4 were all used. Also calculated was the percentage of subjects who fulfilled quality criteria for a good beginning of test, as proposed by ATS1 (Vbe <150 mL or 5% FVC) and the European Respiratory Society (ERS)7 [Vbe <100 mL or 5% FVC]. The end-of-test criteria was not assessed independently because the end-of-test volume and forced expiratory time are not provided by the software.

Spirometric variability was analyzed with longitudinal models (generalized estimation equation) using as the dependent variable one indicator of reproducibility (the COV or the percentage difference between the best two maneuvers), and as independent variables, the phase of the study (indicator variables from 1 to 7), spirometry order (indicator variables from 1 to 7), height, age, gender, and technician. Similar models were designed to assess intertest (intrasubject with 6-month difference) variability adjusting by study phase, age, gender, height, and technician. Calculations were done with the help of Stata Statistical Software (Stata Corporation; College Station, TX).


   Results
TOP
 Abstract
 Introduction
 Materials and Methods
 Results
Discussion
 Appendix
 References
 
Biannual assessments were done in the spring (April to June) and fall (October to November). The first cross-sectional evaluation was done during spring 1996, and the seventh was done in spring 1999. The mean number of spirometric tests done per subject was 4.5 (range, 1 to 7) in 3,347 different children, with a total of 15,563 spirometric tests. The population had a mean age of 10.6 years (SD, 1.2) ranging from 7 to 16 years, with an equal distribution of male and female subjects (Table 1 ). Children increased in weight, height, and pulmonary function during the study, whereas spirometric function expressed as a percentage of reference values was held constant.


View this table:
[in this window]
[in a new window]

  		Table 1.. Patient Characteristics*

 
Table 2 shows measurements of several indicators of intratest spirometric variability and quality during the study. In general, spirometric variability decreased and quality increased during the study. A total of 15,563 spirometric tests were done during the study. A total of 14,124 spirometric tests (90.7%) had three acceptable maneuvers according to the ATS criteria, 818 tests (5.3%) had two acceptable maneuvers, 410 tests (2.6%) had one acceptable maneuver, and only 215 tests (1.4%) had no acceptable maneuvers. Reproducibility of the best two tests to < 200 mL was achieved for FVC and FEV1 in 15,140 tests (97.3%), for FVC or FEV1 in 306 tests (2.0%), and for neither only in 121 tests (0.8%). However, grade A for flow was obtained in 8,767 tests (56.3%) and for volume in 1,422 tests (9.8%). Poor-quality tests decreased progressively as the study advanced. For example, during the first evaluation, 155 tests (8.4%) had no acceptable maneuvers, vs 4 tests in the seventh evaluation (0.2%). Some 121 tests (6.5%) in the first evaluation did not achieve reproducibility in both FVC and FEV1 vs 59 tests in the seventh evaluation (0.7%). Similar information is obtained using other indicators, such as the COV or the percentage difference between the two best maneuvers.


View this table:
[in this window]
[in a new window]

  		Table 2.. Spirometric Quality and Intratest Variability per Evaluation*

 
Table 3 shows percentiles found in the studied population for several indicators of acceptability or reproducibility. Note that Vbe was < 5% and 150 mL in almost 95% of the tests, but only in 75% was it < 100 mL. However, 95% of the children could achieve reproducibility for FVC and FEV1 at the 5% level and < 150 mL, but for PEFR only to 15% and < 800 mL/s, as suggested by Enright et al.8 Table 4 shows the percentage of subjects fulfilling several quality criteria proposed by the ERS and ATS for adults and also those proposed by Enright et al8 for children.


View this table:
[in this window]
[in a new window]

  		Table 3.. Percentiles of Quality Measurements (n = 15,566 observations)*

 

View this table:
[in this window]
[in a new window]

  		Table 4.. Percentage of Children Fulfilling Several Criteria of Quality Spirometry Number

 
Spirometric variability tended to be higher and quality lower in girls than in boys. Overall, 90.6% of the tests in girls had three acceptable maneuvers vs 92% in boys; 0.97% of tests in girls had poor reproducibility in FVC and FEV1 vs 0.90 in boys. Boys reached A grade tests more frequently than girls for flows (58.7% vs 53.8%) and volumes (12.7% vs 7.1%), according to the criteria of Enright et al.5 This was also confirmed in multivariate models.

Adjusting for the rest of the variables, including size, age, and gender, a 6% increase of FEV1 in the seventh evaluation was observed, which could be attributed to training; however, one technician obtained on average values 8% higher and another 6% lower than the rest, adjusting by the rest of the variables. Spirometric reproducibility of FEV1, FVC, FEF25–75, PEFR, and other instantaneous flows such as maximal expiratory flow when 50%, 25%, 75% of the FVC remains to be expired, increased in the study after the first evaluation. Variability adjusted by study phase (generalized estimation equation model) was lower in boys than in girls for the main measurements: FEV1, FVC, and PEFR. Similar results were obtained using the flow and volume scores of Enright et al5 as the dependent variables. In longitudinal models adjusted by age, height, phase of study, and technician, the group with the least intratest variability was that made up of tall young boys. In longitudinal multivariate models, the intratest variability of FEV1 decreased with study phase (time), but additional association with spirometry order was not present. However, study phase and spirometry order contributed significantly to intratest variability of FVC and instantaneous flows, decreasing with time and number of spirometric tests done, likely due to the training of children and technicians.

Technicians contributed substantially to intratest and intertest variability. When consecutive spirometric tests were done by the same technician, variability decreased significantly. Those technicians who performed better, obtained higher spirometric values despite adjusting simultaneously by height, weight, gender, phase of study, and order of spirometry in a longitudinal model. Compared with the technician who performed more studies, the mean estimated difference of the rest of technicians was + 15.9 mL for FEV1 and + 34.6 mL for FVC, with 75% of the values between - 15 mL and + 63 mL for FEV1 and - 4 to + 82 for FVC. Twelve technicians did 71% of all tests, and their performance did not differ significantly from the rest.

Intertest variability was assessed in 10,097 spirometric tests done in 2,746 subjects with repeated tests with a 6-month difference. In this group, the mean number of tests was 3.7. Table 5 shows the 6-month intertest (intrasubject) reproducibility, estimated by using the Pearson correlation coefficient, the intraclass correlation coefficient, and the COV for all tests done. The intraclass correlation coefficient for FEV1 and FVC was > 0.8, and the COV was approximately 5% and changed little, excluding outliers. Variability increased between the fifth and sixth evaluations, but then decreased again (Table 5) . Reproducibility was poorer for PEFR but also improved considerably during the study.


View this table:
[in this window]
[in a new window]

  		Table 5.. Intertest (6 Month Difference) Correlation Coefficients and COV

 
Mean spirometer temperature during testing was 21.6°C (SD, 3.1), with 90% of the measurements between 17°C and 27°C. Spirometer temperature correlated negatively with spirometric measurements (body temperature and pressure, saturated) in multiple regression models including height, weight, gender, and age: on average, - 7.7 mL (95% confidence interval, - 6 to - 9 mL) for FEV1, and - 7.5 mL (95% confidence interval, - 6 to - 9 mL) for FVC per degree Celsius.9

The FEV1 was on average 103.4% (SD, 113.1) of that predicted by the third National Health and Nutrition Evaluation Survey for Mexican Americans,10 and 100.6% (SD, 12.0) of that predicted for Mexicans;11 whereas similar numbers for FVC were 104.6% (SD, 13.5) and 99.9% (SD, 11.7).


   Discussion
TOP
 Abstract
 Introduction
 Materials and Methods
 Results
 Discussion
Appendix
 References
 
The ATS recommends several strategies to improve the quality of spirometric tests, beginning with the selection of the type of spirometer used.1 In addition, spirometers should be calibrated routinely using a volume "gold standard"; however, there is no "gold standard" for technician performance, for procedures, and for the interaction between subject and technician. Therefore, the main purpose of spirometry quality control is to reduce variability or increase reproducibility (reliability). The closeness between the two best maneuvers for main spirometric variables is then an important indicator of spirometric quality. The percentage difference and the COV between the two best measurements (with a difference of minutes or seconds) were used for intratest reliability. The former is the most common indicator used in spirometry field testing, while the second is better known in the literature on statistics. Both measurements decreased with time and were smaller in boys—especially younger and taller ones—and give similar information, one being a linear function of the other. The criteria of Enright et al5 included in the Lung Health Study were more difficult for children to achieve, in part because the spirometers used assess test quality and provide messages only for ATS criteria.

It is also important to assess reproducibility periodically to ascertain if quality was maintained, a difficult issue in longitudinal studies; however, children and their lung function grow between two consecutive tests (6 months) and therefore spirometric measurements differ even with good quality control. Despite lung growth, the correlation between two consecutive tests will be higher and the COV lower in better-performed tests. Intertest reproducibility was maintained during the study, except for a transient drop during the fifth phase that coincided with an episode of high air pollution. During this episode, some of the technicians participated in an epidemiologic surveillance team that returned to the evaluation of students at a later date. Due to the time constraints of evaluation, it is likely that a group of students was evaluated quickly in order to finish before the summer break.

Children were able to fulfill the ATS criteria of acceptability used with adults, including the Vbe, end-of-test criteria, and an expiration lasting for at least 6 s. Almost 91% of the tests evaluated had three acceptable maneuvers, and only 1.4% obtained no acceptable maneuvers. ATS requirements for adults can be achieved in children with few modifications as suggested by Enright et al.8 It was noticeable that 93.9% of the population was able to reproduce FEV1 and FVC to < 150 mL or 5%, slightly better than previously reported.8 Approximately 95% of the children studied fulfilled each of the standards proposed by Enright et al8 for children, but not all of them together. For example, 92.3% of the tests in the subjects have an extrapolated volume < 5% of FVC, 94.9% have an absolute difference between the two highest FVC values (dFVC) < 200 mL and 5%, 94.9% have an absolute difference between the two highest FEV1 values (dFEV1) < 200 mL and 5%, and 94.0% have a dPEFR < 1,000 mL/s and 5%. However, only 79.5% of the subjects fulfilled all those four criteria simultaneously. Neither the forced expiratory time, PEFR time, nor the end-of-test volume could be evaluated quantitatively because they are not provided in the database of the spirometers, but it is likely that the percentage of children complying simultaneously with all proposed quality criteria will be low. It is clear that specific quality criteria for spirometry in children must be agreed on.12

The impact of technicians on the pulmonary function in longitudinal multivariate models was important. In this study, two technicians with extreme performance produced estimated differences in FEV1 of 200 mL (approximately 12%), adjusted for the rest of the variables, a magnitude that is clinically significant, although three fourths of the technicians differ < 70 mL on average. Technician training and permanent evaluation is therefore essential; in addition, its contribution should be taken into account during analysis.

The quality of spirometry increased from the second phase of the present project, due to learning by both children and technicians, as well as a better interaction between both. Ideally, the best level of performance in technicians and children should be attained in a pilot phase, before obtaining data, but in this study that was unfeasible. Another priority in prolonged studies is keeping the interest of people who are taking part: as enthusiasm tends to wean, so does spirometric quality. This goal was also achieved, despite a minor decrease in spirometric quality during the fifth phase that was solved afterwards.

Clinical spirometry in children also requires quality assurance. Although the specific levels of reproducibility achievable in routine pediatric care have to be determined, the issues discussed in the present document are relevant to clinical practice including the importance of technician training, evaluation and supervision.


   Appendix
TOP
 Abstract
 Introduction
 Materials and Methods
 Results
 Discussion
 Appendix
References
 
Other members of the EMPECE study group are as follows: Rocio Chapela, MD, Jaime Villalba, MD, and, in charge of spirometry quality field supervision, Margarita Rojas, MD, and Minerva Catalán from the National Institute of Respiratory Diseases; Isabelle Romieu, MD, from the National Institute of Public Health; and Teresa Fortoul, MD, from the National Autonomous University of Mexico.


   Acknowledgements
 
We are indebted to the technicians and teachers, but especially to the children who participated and their parents.


   Footnotes
 
Abbreviations: ATS = American Thoracic Society; COV = coefficient of variation; dFEF25–75% = percentage difference between the two highest mid-expiratory flow rates; dFEV1 = absolute difference between the two highest FEV1 values; dFEV1% = percentage difference between the two highest FEV1 values; dFVC = absolute difference between the two highest FVC values; dFVC% = percentage difference between the two highest FVC values; dPEFR = absolute difference between the two highest peak expiratory flow rates; dPEFR% = percentage difference between the two highest peak expiratory flow rates; ERS = European Respiratory Society; FEF25–75 = forced mid-expiratory flow rate; FET100%VC = average time to reach vital capacity; PEFR = peak expiratory flow rate; Vbe = back-extrapolated volume

This work was presented at the American Thoracic Society Meetings held in May 2001.

Supported by CONACYT (National Council of Science and Technology of Mexico), the Ministry of Health, the National Institutes of Respiratory Diseases and Public Health, the CONSERVA Foundation and the following pharmaceutical laboratories: Ciba, Glaxo, Promeco, Eli Lilly, and Smith, Kline, and Beecham.

Received for publication May 15, 2002. Accepted for publication October 11, 2002.


   References
TOP
 Abstract
 Introduction
 Materials and Methods
 Results
 Discussion
 Appendix
 References
 

  1. . American Thoracic Society (1995) Standardization of spirometry, 1994 update. Am J Respir Crit Care Med 152,1107-1136[ISI][Medline]
  2. Nelson, SB, Gardner, RM, Crapo, RO, et al Performance evaluation of contemporary spirometers. Chest 1990;97,288-297[Abstract/Free Full Text]
  3. Linn, WS, Solomon, JC, Gong, H, et al Standardization of multiple spirometers at widely separated times and places. Am J Respir Crit Care Med 1996;153,1309-1313[Abstract]
  4. Szklo, M, Nieto, FJ Quality assurance and control. Epidemiology: beyond the basics 2000,343-404 Aspen Publishers. Gaithersburg, MD:
  5. Enright, PL, Johnson, LR, Connett, JE, et al Spirometry in the Lung Health Study: methods and quality control. Am Rev Respir Dis 1991;143,1215-1223[ISI][Medline]
  6. Hnizdo, E, Churchyard, G, Barnes, D, et al Assessment of reliability of lung function screening programs or longitudinal studies. Am J Respir Crit Care Med 1999;160,2006-2011[Abstract/Free Full Text]
  7. Quanjer, PH, Tammeling, GJ, Cotes, JE, et al Lung volumes and forced ventilatory flows: Report Working Party Standardization of Lung Function Tests, European Community for Steel and Coal; Official Statement of the European Respiratory Society. Eur Respir J Suppl 1993;16,5-40[Medline]
  8. Enright, PL, Linn, WS, Avol, EL, et al Quality of spirometry test performance in children and adolescents: experience in a large field study. Chest 2000;118,665-671[Abstract/Free Full Text]
  9. Gilliland, FD, Linn, W, Rappaport, E, et al Effect of spirometer temperature on FEV1 in a longitudinal epidemiological study. Occup Environ Med 1999;56,718-720[Abstract]
  10. Hankinson, JL, Odencrantz, JR, Fedan, KB Spirometric reference values from a sample of the general US population. Am J Respir Crit Care Med 1999;159,179-187[Abstract/Free Full Text]
  11. Perez-Padilla, JR, Regalado, J, Rojas, M, et al Lung function growth of inhabitants of Mexico City as determined by a cross sectional study [abstract]. Am J Respir Crit Care Med 2001;163,A883
  12. Arets, HG, Brackel, HJ, van der Ent, CK Forced expiratory manoeuvres in children: do they meet ATS and ERS criteria for spirometry? Eur Respir J 2001;18,655-660[Abstract/Free Full Text]




This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF) Free
Right arrow Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Article Archive
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via ISI Web of Science (3)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Pérez-Padilla, R.
Right arrow Articles by Olaiz, G.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Pérez-Padilla, R.
Right arrow Articles by Olaiz, G.

HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS