Variability in the Classification of Radiographs Using the 1980 International Labor Organization Classification for Pneumoconioses

¹ From the Department of Occupational/Environmental Medicine, Washington Hospital Center, Washington, DC
² From the George Washington University, Washington, DC
³ From the University of California at San Francisco, San Francisco General Hospital, San Francisco, CA
⁴ From the Department of Community and Preventive Medicine, Allegheny University of Health Sciences, MCP/Hahneman School of Medicine, Philadelphia, PA
⁵ From the University of Alberta Faculty of Medicine, Edmonton, AR, Canada
⁶ From the Division of Occupational and Environmental Medicine, Department of Environmental Health, University of Cincinnati, Cincinnati, OH
⁷ From the Mt. Saint Joseph Hospital, Vancouver, BC, Canada

Laura S. Welch, MD, Occupational Environmental Medicine, Washington Hospital Center, 100 Irving St, NW, EB1114, Washington, DC 20010-2975

This study describes the extent of agreement in classification of chest radiographs using the International Labor Organization (ILO) classification among six readers from the United States and Canada. A set of 119 radiographs was created and read by three Canadian and three US readers. The two ratings of interest were profusion (scored from 0/ to 3/+) and pleural abnormalities consistent with pneumoconiosis (scored with the ILO system, then collapsed into a yes/no). We used a number of approaches to evaluate interreader agreement on profusion and pleural changes, determining concordance, observed agreement, kappa statistic, and a new measure to approximate sensitivity and specificity. This study found that five of six readers had good fair to good agreement for pleural findings and for profusion as a dichotomous variable ( 1/0 vs 0/1) using the kappa statistic, while a sixth reader had poor agreement. We found that concordance, expressed as percent agreement, was higher for normal radiographs than for ones that showed disease, and describe the use of the kappa statistic to control for this finding. This analysis adds to the existing literature with the use of the kappa statistic, and by presenting a new measure for "underreading" and "overreading" tendencies.

Key Words: asbestos • pneumoconiosis • radiograph • sheet metal worker

Submitted on September 30, 1997
Accepted on April 21, 1998

This article has been cited by other articles:

				L. S. Welch, E. Haile, J. Dement, and D. Michaels Change in Prevalence of Asbestos-Related Disease Among Sheet Metal Workers 1986 to 2004 Chest, March 1, 2007; 131(3): 863 - 869. [Abstract] [Full Text] [PDF]

				R. M. Ross The Clinical Diagnosis of Asbestosis in This Century Requires More Than a Chest Radiograph Chest, September 1, 2003; 124(3): 1120 - 1128. [Abstract] [Full Text] [PDF]

				D R Baldwin, P Gannon, P Bright, D T Newton, A Robertson, K Venables, B Graneek, R D Barker, A Cartier, J-L Malo, et al. Interpretation of occupational peak flow records: level of agreement between expert clinicians and Oasys-2 Thorax, October 1, 2002; 57(10): 860 - 864. [Abstract] [Full Text] [PDF]