|
|
||||||||
Guest Access | Sign In via User Name/Password |
|||||||||
* From the Memorial Sloan-Kettering Cancer Center (Drs. Bach, Elkin, Kattan, and Begg), New York, NY; Istituto Nazionale Tumori (Dr. Pastorino), Milan, Italy; the Weill Medical School (Dr. Mushlin), Cornell University, New York, NY; and the International Association for Research on Cancer (Drs. Bach and Parkin), Lyon, France.
Correspondence to: Peter B. Bach, MD, FCCP, Memorial Sloan-Kettering Cancer Center, 1275 York Ave, Box 221, New York, NY 10021; e-mail: bachp{at}mskcc.org
| Abstract |
|---|
|
|
|---|
Design/setting/patients: Model derivation was based on analyses of the placebo arm of the Carotene and Retinol Efficacy Trial. Model validation was based on analyses of three other longitudinal cohorts.
Measurements: Observed and predicted number of deaths due to lung cancer.
Results: In internal validation, the model was highly concordant and well calibrated. In external validation, the model predictions were similar to what was observed in all of the validation analyses. The predicted and observed deaths within 6 years were very similar when assessed in the Johns Hopkins Hospital trial of chest radiography and sputum cytology screening (176 predicted, 184 observed, p = 0.53), the Memorial Sloan-Kettering Cancer Center trial of chest radiography and sputum cytology screening (108 predicted, 114 observed, p = 0.57), and the National Health and Nutrition Evaluation Survey part I (24 predicted, 21 observed, p = 0.52).
Conclusions: The number of lung cancer deaths in a population of current or former smokers can be accurately predicted, making model-based evaluations of prevention and early detection interventions a useful adjunct to definitive randomized trials. We illustrate this potential use with a small example.
Key Words: Cancer screening CT logistic models lung cancer risk assessment
| Introduction |
|---|
|
|
|---|
It is generally accepted that the only valid end point for evaluations of cancer screening programs is cancer-specific mortality.5 As such, most investigators agree that the randomized trial will produce a definitive estimate of the impact of screening, because it will allow the lung cancer mortality rate among individuals who are screened with LDCT to be directly compared with the lung cancer mortality rate of a similar group ("the controls") who are not screened with LDCT. Investigators disagree about studies that have not included a control group, in which stage at diagnosis and survival after diagnosis are the primary end points; in the context of a screening program, changes in these measures may not be predictive of changes in cancer-specific mortality, due to phenomena such as "lead time" and "length time" bias.6
What has not been considered is whether this "gold standard" outcome for screening evaluationthe lung cancer mortality ratecan be examined in the groups who have been screened, to determine if this rate is lower than it would have been in the absence of screening. To perform such a comparison would require that the lung cancer mortality rate that would have occurred in the absence of screening could be accurately anticipated. In this article, we describe a model that can be used to calculate the expected number of lung cancer deaths that will occur within a population of current and former smokers in the absence of screening or prevention intervention.
There is precedent for using nonrandomized comparisons to evaluate the impact of screening strategies. Investigators789 concluded that the Papanicolou test reduced the cervical cancer mortality rate when rates of cervical cancer incidence and mortality were reduced after introduction of the test. However, in these studies,789 the magnitude of the screening impact was very large. In contrast, investigators1011 concluded that newborn screening for neuroblastoma did not reduce mortality from the disease by comparing neuroblastoma mortality rates in populations that were screened to other populations that had not been screened. Lung cancer differs from these examples in that investigators seek to demonstrate a more modest impact than that demonstrated for the Papanicolou test or for colonoscopy. However, lung cancer is also unusual in that the major risk factorscigarette smoking and agehave a very strong impact on risk, and the degree of exposure is easily obtained from an interview.12
| Materials and Methods |
|---|
|
|
|---|
The data from CARET are highly appropriate for deriving our model. They are rich in subjects and events, and the risk factors and outcomes were evaluated and recorded scrupulously. Moreover, the population of subjects in CARET are relevant to our purposes, in that they are drawn from six geographically diverse areas of the United States (Seattle WA, Portland OR, Irvine CA, Baltimore MD, New Haven CT, San Francisco CA), they consist only of volunteers in a clinical trial of cancer prevention, and they possess the smoking risks that make them strong candidates for lung cancer screening and prevention.
Our approach to model development and initial testing paralleled that which we used to develop a model of an individuals risk of having lung cancer diagnosed.12 First, the data in CARET were divided into separate person-time periods. The beginning of each time period was defined by date of an encounter with a study coordinator (either initial or follow-up), and the end of the time period was defined either by the date of the outcome (death due to lung cancer) or by the date of a censored event (subsequent follow-up encounter, death due to another cause, end of the data). These person-time periods were then coded as either commencing within 1 year of study entry, or commencing after that point. The data were dichotomized in this manner to capture any risk attenuation that was initially present among study entrants due to their being asymptomatic at the time of enrollment.
Our validation involved subjects in the following three cohorts (Table 1 ): the National Health and Nutrition Evaluation Survey (NHANES) I6; the Memorial Sloan-Kettering Cancer Center (MSKCC) randomized trial of chest radiograph and sputum cytology screening15; and the Johns Hopkins Hospital (JHH) randomized trial of chest radiograph and sputum cytology screening.16 These validation cohorts were comprised of individuals who were eligible for and elected to enroll in longitudinal health studies, making them well suited to our purposes. There were also some shortcomings to the use of these cohorts. The NHANES I cohort is small, and the subjects were not asked about asbestos exposure; we assumed in our analyses that none of the subjects had been exposed to asbestos. The MSKCC and JHH cohorts included male subjects only; moreover, all subjects were screened for lung cancer in the context of the study. Because it is widely accepted that the screening interventions in these studies did not alter the lung cancer mortality rates, we assumed that the lung cancer mortality rates in these studies were similar to rates that would have been observed in the absence of screening.61516
|
To illustrate how the model might be used to evaluate single-arm studies of lung cancer screening, we applied the model to an ongoing single-arm study being conducted in Milan Italy, in which subjects have been annually screened with low dose CT. This study has been described in detail elsewhere.3 The study began enrolling study subjects in 2000 with a prespecified plan for follow-up and cause-of-death ascertainment. Follow-up has been accomplished through annual follow-up scans, telephone contact of those subjects who did not appear for follow-up scans, and searches of death registries for the Lombardy region of Italy. Through these means, all but one of the study subjects had vital status ascertained in 2003 or 2004; the one outstanding subject had follow-up only through December 2001, at which point he was alive and free of a lung cancer diagnosis. For the 16 study subjects who died, 14 medical records were obtained and reviewed in order to assign cause of death. For the remaining two decedents, cause of death was ascertained through reviews of death certificates, the Lombardy cancer registry, and health population files. All subjects who died after a diagnosis of lung cancer were assigned lung cancer as a cause of death in our analysis.
Statistical Analysis
Model Development and Internal Validation:
Cox proportional hazards regression was used to estimate the multivariable relations between the risk factors and the risk of lung cancer death. The proportional hazards assumption that the hazard ratios were constant over time was verified by tests of correlations with time and examination of residual plots. Continuous predictors (age, duration of quitting, duration of smoking, number of cigarettes smoked per day) were fit with restricted cubic splines to allow for nonlinear and nonmonotonic effects; the knots separated quartiles of the data. Sex, asbestos exposure, and study entry were treated as categorical variables. Model discrimination was assessed by the concordance index after tenfold cross-validation, a form of resampling that reduces the bias that is introduced when the model is tested on the data from which it was derived.17181920 Internal calibration was assessed using a calibration plot. In this plot, the cohort was divided into 10 equally sized groups by level of predicted risk. Then, the observed probabilities of lung cancer death were compared to those predicted both in the empiric data, and after tenfold cross-validation.
External Model Validation and Applied Example:
The number of lung cancer deaths predicted by the model was compared to the number of lung cancer deaths observed in the validation and Milan cohorts. For each individual and for each year of the study, we generated estimates of the probability of death from lung cancer. During the first year of the study, the individuals risk was based on their risk factors at the time of study entry, including the risk attenuation estimate, a factor included to reflect the lower immediate risk of death for individuals who are healthy enough to register in a clinical trial. In subsequent years, risk was calculated from incremented risk factors; age increased each year for all subjects, and years of smoking or years of quitting increased for individuals depending on their smoking status at entry. To determine the number of lung cancer deaths expected in each period, we summed the individual probabilities for those subjects who were still being followed up in that time period of the study. In other words, each individual contributed to the group estimate only over the duration of his or her follow-up.
The predicted and observed number of lung cancer deaths were compared in each of the studies individually; for the validation step, all three validation cohorts were combined. To test whether the numbers differed, we used the
2 test for Poisson distributed variables [(O-E)] ![]()
![]() |
2 test. These analyses of the CARET data were approved by the Institutional Review Boards at the Fred Hutchinson Cancer Research Center, Seattle, WA, and MSKCC, New York, NY. Model development was conducted with appropriate software (S-Plus software, version 2000 Professional; Insightful; Seattle, WA) with additional functions (called Design).17 Predicted and observed mortality curves were generated and compared with a statistical software package (STATA 8.1; StataCorp; College Station, TX). The model formula is available by request from the author.
| Results |
|---|
|
|
|---|
|
|
20%. In the individual cohorts, the power was substantially less; for a difference of the same magnitude, the power in the JHH cohort was 74%, in the MSKCC cohort was 54%, and in the NHANES I cohort was 19%.
|
|
| Discussion |
|---|
|
|
|---|
We pursued this study because in lung cancer there are easily observable factors that very strongly predict the risk of this disease and death due to it (age and detailed smoking history). Our analyses were also justified by the availability of a large high-quality data set for model derivation (the data from CARET). In CARET, the population under study was similar to the highly selected populations that are included in trials of lung cancer screening (middle-aged and elderly people with a long-term smoking history who lack symptoms of lung cancer). Because this set of favorable circumstances are not applicable to any of the other common cancers, our approach may not be widely generalizable beyond lung cancer.
Investigators might consider applying our model to their studies of lung cancer screening or prevention if (and only if) their studies have the following features: (1) the characteristics and risk factors of all subjects are ascertained at study entry; (2) the outcomes of all subjects are determined, including those subjects who only inconsistently received the study intervention(s); and (3) all subjects who were asymptomatic at study entry are included in all analyses, even if they were discovered to have advanced lung cancer during the initial screening process. There may be many ongoing studies of LDCT that have these features, and these studies may collectively include tens of thousands of participants and person-years of follow-up. As such, these studies might be used to generate a preliminary estimate of the impact of lung cancer screening with low-dose CT on lung cancer mortality, at least until more definitive results from the National Lung Screening Trial are available.621
We caution that evaluating outcomes of individuals in single-arm studies of screening or prevention using our model is an approach potentially susceptible to a variety of biases, some which affect all models, and some which are unique to our model. In general, a predictive model is typically less accurate in a new group of subjects than it is when originally described due to differences between the cohorts that were used to derive the model and the cohorts to which it is applied.2223 For our model, the cohort (CARET) that was used for derivation was assembled and evaluated during the late 1980s and early 1990s, and the cohorts used for evaluation were enrolled in studies more than a decade earlier. Cohorts enrolled in current studies of screening and prevention began smoking many years after the cohorts included in our derivation and validation cohorts, and may have been exposed to cigarettes with different nitrate and tar composition. Our model has not been sufficiently validated in women either, given that neither the JHH or MSKCC studies enrolled women. With the rising incidence of the disease in women, this is an important shortcoming.24
Subjects enrolled in CARET and in our validation studies faced a competing risk of death due to causes other than lung cancer; in many cases, these causes of death (particularly cardiovascular death) are also closely linked to the risk factors that confer an excess risk of lung cancer death (ie, more advanced age, greater degree of smoking exposure). Medical progress has successfully reduced the risk of death due to many of these smoking-related diseases. As a result, in contemporary cohorts, the observed lung cancer mortality rate may actually be higher than it would have been at the time CARET was conducted. This phenomenon may cause the model to underpredict lung cancer mortality rates, so the direction of this bias will not cause studies of screening or prevention to appear falsely beneficial.
The excellent performance of our model in our validation analyses does suggest that this approach to the assessment of screening and prevention interventions should receive further consideration. Before being assessed in randomized studies, new therapeutic agents in cancer are evaluated by comparing the overall response rate of a group of subjects who receive the new agent to the overall response rate of a set of historical controls; these latter individuals provide the "expected" response rate. In the evaluation of new screening and prevention strategies in lung cancer, this type of comparison has not been performed, because no suitable group of historical controls exists. Our statistical model of lung cancer mortality might be useful for evaluating lung cancer prevention and early detection strategies, because it provides an analog to "historical controls." The added advantage of our approach in this context is that each person serves as his or her control, an important feature when the risk of the disease varies broadly across subjects.
| Acknowledgements |
|---|
| Footnotes |
|---|
These analyses were supported by a grant from the Steps for Breath Fund at MSKCC, New York, NY, and by RO1CA090226 (both to Dr. Bach).
Received for publication June 7, 2004. Accepted for publication July 20, 2004.
| References |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
P. B. Bach, J. R. Jett, U. Pastorino, M. S. Tockman, S. J. Swensen, and C. B. Begg Computed Tomography Screening for Lung Cancer Reply JAMA, August 1, 2007; 298(5): 515 - 516. [Full Text] [PDF] |
||||
![]() |
P. B. Bach, J. R. Jett, U. Pastorino, M. S. Tockman, S. J. Swensen, and C. B. Begg Computed Tomography Screening and Lung Cancer Outcomes JAMA, March 7, 2007; 297(9): 953 - 961. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. Siemiatycki Synthesizing the Lifetime History of Smoking Cancer Epidemiol. Biomarkers Prev., October 1, 2005; 14(10): 2294 - 2295. [Full Text] [PDF] |
||||
![]() |
J. L. Mulshine New Developments in Lung Cancer Screening J. Clin. Oncol., May 10, 2005; 23(14): 3198 - 3202. [Abstract] [Full Text] [PDF] |
||||
Read all eLetters
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |