Chest, Vol 105, 710-714, Copyright © 1994 by American College of Chest Physicians

ARTICLES

Assessment of interrater and intrarater reliability in the evaluation of metered dose inhaler technique

SL Gray, AC Nance, DM Williams and CC Pulliam
University of North Carolina, School of Pharmacy, Chapel Hill.

STUDY OBJECTIVE: To determine if a training session using videotaped metered dose inhaler (MDI) performances can result in high interrater and intrarater reliability of five evaluators assessing MDI technique. DESIGN: Five evaluators (three pharmacists, two pulmonary fellows) were trained to evaluate MDI technique during a 2-h training session. The training session consisted of verbal instruction and practical experience in evaluating MDI technique using video-taped MDI performances of six nonstudy subjects. After the training session, the evaluators independently observed the same videotaped MDI demonstrations of 14 subjects on two occasions separated by a 7- to 10- day interval. Interrater and intrarater reliability was determined for individual steps by calculating percent agreement and intraclass correlation (ICC) coefficient. RESULTS: Interrater. The interrater reliability for individual steps ranged from 29 to 86 percent (ICC coefficient = 0.13 to 0.81). Steps in which evaluators were in agreement for less than 9 of the 14 subjects were shaking the inhaler before inhalation, exhaling, continuing to inhale slowly, and adequate breath hold. Intrarater: The overall percent agreement by step ranged from 74 to 97 percent. Exhaling to functional residual volume (76 percent) and continuing to inhale slowly and deeply (74 percent) had the lowest overall agreement between the first and second observation day. The consistency of evaluating a step between the two observation days varied considerably depending on the step and evaluator. CONCLUSIONS: High interrater and intrarater reliability in MDI evaluation is difficult to obtain. Clinicians and researchers involved in MDI evaluation and education should be trained to achieve consistency. A single training session using videotaped MDI demonstrations was not adequate in achieving consistency among evaluators. To improve accuracy of research results, researchers should include at least two evaluators to assess MDI technique or take other measures to show and report reliability.

This article has been cited by other articles:

				M Lee-Wong and P H Mayo Results of a programme to improve house staff use of metered dose inhalers and spacers Postgrad. Med. J., April 1, 2003; 79(930): 221 - 225. [Abstract] [Full Text] [PDF]

				S. Epstein, A. Maidenberg, D. Hallett, K. Khan, and K. R. Chapman Patient Handling of a Dry-Powder Inhaler in Clinical Practice Chest, November 1, 2001; 120(5): 1480 - 1484. [Abstract] [Full Text] [PDF]

				D. M. Taylor, T. E. Auble, W. J. Calhoun, and V. N. Mosesso Jr. Current Outpatient Management of Asthma Shows Poor Compliance With International Consensus Guidelines* Chest, December 1, 1999; 116(6): 1638 - 1645. [Abstract] [Full Text] [PDF]

				R. Ramsey-Goldman and K. W Hayes Ensuring reliability of clinical assessments for SLE trials Lupus, October 1, 1999; 8(8): 671 - 676. [Abstract] [PDF]