| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
Guest Access | Sign In via User Name/Password
|
|
|
|||||||
Guest Access | Sign In via User Name/Password |
|||||||||
* From the Departments of Radiation Oncology (Drs. Machtay and Glatstein) and Thoracic Surgery (Dr. Kaiser), Hospital of the University of Pennsylvania, Philadelphia, PA.
Correspondence to: Mitchell Machtay, MD, Department of Radiation Oncology, 2 Donner Bldg, Hospital of the University of Pennsylvania, 3400 Spruce St, Philadelphia, PA 19104
Key Words: lung cancer • meta-analysis • postoperative radiotherapy
 |
Introduction |
|---|
 TOP Introduction References |
|---|
There is a growing obsession within the medical community and the public for swift and succinct evidence-based decisions. However, the presumption of many of those who advocate this view is that only meta-analyses of prospectively randomized trials—however flawed—provide useful information. The meta-analysis of postoperative radiotherapy (PORT) in non-small cell lung cancer (NSCLC) published recently in Lancet1 well exemplifies this way of thinking. "It should be compulsory reading for CEOs of hospitals. There is a very clear message here," said the director general of Britain's Cancer Research Campaign to Associated Press reporters. He congratulated the PORT researchers for "determining" that PORT for NSCLC not only failed to improve survival but actually worsened survival. The publicity generated by the PORT meta-analysis caused unwarranted fear among many lung cancer patients, who already have enough to worry about.
The popular and academic media have not adequately addressed a real discussion of the true scientific limitations of this study and of the questionable validity of meta-analyses in general. Although performed with statistical rigor and considerable effort on the part of the researchers to obtain individual, updated patient data, the Lancet meta-analysis is scientifically flawed. This is particularly with respect to the "sensational" conclusion that radiotherapy (XRT) causes more deaths than it prevents in lung cancer patients.
The first problem with meta-analysis, which the PORT researchers admirably tried to solve, is that it presumes that the authors have all of the data from all randomized studies, published or not. There is a well-known bias against negative studies ever seeing the light of publication. The PORT researchers state that they exhaustively searched for all published and unpublished data. Nonetheless, two randomized trials2 3 were excluded from their meta-analysis; interestingly, neither of these excluded trials showed evidence of a detrimental survival effect of PORT and they actually suggested a trend toward a benefit for node-positive patients. The PORT researchers have not adequately explained why these trials were excluded.
The second problem relates to an ongoing "deification" of randomized trials and the meta-analytic process itself. The obsession with randomized studies assumes that they are all equally reliable and thus it is valid to combine them into one analytic exercise. Of course, this is mixing apples with oranges, resulting in a fruit cocktail. Although one may describe fruit cocktail in detail, one cannot analyze it. Randomized data are important, but they are not necessarily the only basis for decision making. Despite a consumer desire to simplify the thinking process to a simple computer algorithm that will "decide," there is an obvious reality that the practice of medicine remains not a science, but an art that incorporates a scientific base.
In the PORT meta-analysis, a large number of patients (approximately 26% of the entire sample) underwent resection (with negative margins) and were found to have negative regional lymph nodes (American Joint Committee on Cancer [AJCC] stage I disease). There is no rational scientific basis for PORT in these patients, as they are known to have a low risk of local-regional recurrence.4 5 Few well-trained radiation oncologists would routinely advise PORT in these patients. At the other extreme, patients with positive mediastinal nodes (AJCC stage IIIA disease) have a high risk of local-regional recurrence and are more likely to benefit from PORT, as has been shown in numerous retrospective studies.6 7 8 9 10 Patients with negative mediastinal nodes but with positive hilar or peribronchial nodes (AJCC stage II disease) have an intermediate risk of local-regional recurrence, which may depend heavily on treatment-related factors, which will be discussed. All three stages of NSCLC were included in the PORT meta-analysis to arrive at the conclusion widely quoted in the media. This is analogous to performing a meta-analysis on the utility of coronary artery bypass grafting for every type and degree of heart disease.
Second, the value of postoperative adjuvant treatment depends critically on the adequacy of the surgery itself. Those who advocate the meta-analysis assume that the surgery is somehow generic and all of a similar quality. That may be relatively true in breast cancer surgery, in which the definition of a modified radical mastectomy is reasonably well standardized. However, in lung cancer, as shown in a recent report in CHEST,11 there is enormous variability in the surgery performed. This study suggested that outcome is significantly improved when lung cancer surgery is done by a board-certified thoracic surgeon as opposed to a general surgeon.11 It is likely that most lung cancer resections throughout the world are performed by general surgeons, or by thoracic surgeons whose primary activity is cardiac surgery rather than thoracic oncologic surgery.
Thoracic oncologic surgeons perform a detailed and planned mediastinal nodal dissection, as was carried out in the well-known Lung Cancer Study Group trial.12 This study nicely showed that PORT added little to the outcome of patients with N1 disease (there were relatively few patients in the study with N2 disease) who have had a complete mediastinal "clean-out." However, one cannot extrapolate from their study, because the typical surgeon (whether within or outside protocols) randomly samples only a few or even no mediastinal lymph nodes. Because the stage depends largely on the adequacy of the lymph node sampling, the confidence one can have in the stage is inversely proportional to the extent of nodal sampling. There are also likely to be differences in the choice of surgery on the primary tumor (lobectomy vs partial lobe removal vs pneumonectomy) and the method(s) of ensuring clear margins of resection. These crucial variables, any of which can affect the efficacy or toxicity of adjuvant therapy, are often not even mentioned in randomized trials, let alone stratified.
XRT, although probably not as operator-dependent as surgery, is also performed very heterogeneously throughout the world. This is, again, particularly true for lung cancer. In most malignancies, such as breast or colon cancer, the patient's postoperative performance status is usually similar to what it was preoperatively. In contrast, most lung cancer patients have significant underlying COPD, which often worsens after the rigors of a thoracotomy and the resection of substantial lung volume. Not infrequently, this deterioration in performance status and respiratory function precludes PORT. Most radiation oncologists appropriately decline to treat such patients when there is a major concern that irradiation could result in severe and life-threatening complications, even if the patient was "prerandomized" to receive XRT. It is not clear what, if any, criteria were used in the randomized studies of the PORT meta-analysis to exclude particularly fragile patients from the individual studies.
Furthermore, even a relatively fit patient can suffer serious complications of XRT if the treatment is poorly delivered. Most of the trials included in the PORT meta-analysis used outdated radiation sources (cobalt-60) and outdated radiation fields (lateral fields), both of which would be expected to increase the radiation exposure to normal lung tissue and, hence, complications. Cobalt-60 irradiation, used in many if not most of the patients in the PORT meta-analysis, has been shown to have unacceptable complications after pneumonectomy.13 Some trials (including two large unpublished trials) used large daily radiation fraction size or high total radiation doses, which are known to increase the risks of complications.14 15 Almost no information is available in the PORT meta-analysis on the techniques used for shielding of the normal heart and lung; modern techniques of XRT would use CT planning and either customized shielding or multileaf collimation.
It may be that the adverse effect of XRT on survival found in the PORT meta-analysis was a function of poor patient selection and XRT technique in many of the individual randomized trials. The 2-year survival of 48% in the PORT study (which includes many stage I patients) contrasts sharply with our University of Pennsylvania experience (a 64% 2-year survival in a series containing virtually no stage I patients and many patients with positive margins)16 and the Mayo Clinic experience (43% at 4 years in a series made up entirely of stage III patients).10 Of course, these data will be ignored by a meta-analytic study (and therefore most "evidence-based decision makers") because they do not come from randomized trials.
If one were to design an adjuvant prospective randomized study that did not carefully define the surgery, prognostic factors, extent of disease, margins of resection, and details of adjuvant therapy, most physicians would call that study poorly designed and uninterpretable. What makes the reader think that the meta-analytical process will change that? In fact, what evidence is there that the meta-analytic process has the degree of scientific validity its promoters suggest? It began as a method of gathering large numbers of low-risk events in epidemiologic studies, but recently its use has been expanded to randomized studies in which small differences in higher-risk events can be shown to be statistically significant. Whether these statistically significant differences in outcome are clinically meaningful is open to question, particularly when these small treatment-related differences are compared with the much more dramatic effects of patient- and tumor-related prognostic factors. Although it is generally assumed that the process of randomization will equalize all known and unknown factors that affect survival, this can never be guaranteed. How many patients are really necessary to ensure that all the prognostic factors are "smeared out" and not maldistributed?
The disadvantages of meta-analyses have been summarized in previous commentaries over the past 20 years.17 The most consistent criticism of meta-analyses is that they should not be considered to be a replacement for well-designed, well-executed trials performed with the best possible modern treatment. A recent study in the New England Journal of Medicine suggested that large prospective randomized studies performed after the results of "conclusive" meta-analyses frequently arrived at differing, often opposite conclusions.18
Along a similar line, the results of the PORT meta-analysis are reminiscent of another meta-analysis performed 10 years ago. That study, by Cuzick et al,19 showed that postmastectomy irradiation for breast cancer not only failed to improve survival, it worsened survival. Like the PORT lung study, there were innumerable problems with the heterogeneity of patient selection and XRT techniques. In the last several years, multiple well-designed randomized trials have refuted the original conclusions of Cuzick et al19 and have demonstrated a significant survival advantage to postmastectomy irradiation for properly selected and well-treated patients.20 21
One can only infer that belief in the process of meta-analysis is directly proportional to one's bias about the conclusions it has reached. The PORT meta-analysis serves as a strident reminder to radiation oncologists that more clinical and basic science research on minimizing the toxicity of thoracic XRT is needed. More importantly, though, it points out the need for future prospective randomized trials of postoperative XRT that select patients appropriately, account for important prognostic variables, and—perhaps most importantly—incorporate improved adjuvant systemic therapy and modern XRT techniques. We hope that investigators and patients will not accept the flawed conclusions of the PORT meta-analysis as the final word.
|
Footnotes |
|---|
Received for publication December 15, 1998. Accepted for publication March 25, 1999.
|
References |
|---|
|
TOPIntroductionReferences |
|---|
This article has been cited by other articles:
|
|
 |
|
  M. Machtay, J. H. Lee, J. B. Shrager, L. R. Kaiser, and E. Glatstein Risk of Death From Intercurrent Disease Is Not Excessively Increased by Modern Postoperative Radiotherapy for High-Risk Resected Non-Small-Cell Lung Carcinoma J. Clin. Oncol., October 1, 2001; 19(19): 3912 - 3917. [Abstract] [Full Text]
|
|
|
|
 |
|
 R. Arriagada and J.-P. Pignon Is Meta-analysis a Metaphysical or a Scientific Method? Chest, September 1, 2000; 118(3): 832 - 834. [Full Text] [PDF]
|
|
|
|
|
|
M. Machtay, L. R. Kaiser, and E. Glatstein Reality and Meta-analyses Chest, September 1, 2000; 118(3): 835 - 836. [Full Text] [PDF]
|
|
|
|
 |
|
 J. H. Lee, M. Machtay, L. R. Kaiser, J. S. Friedberg, S. M. Hahn, M. G. McKenna, and W. G. McKenna Non-Small Cell Lung Cancer: Prognostic Factors in Patients Treated with Surgery and Postoperative Radiation Therapy Radiology, December 1, 1999; 213(3): 845 - 852. [Abstract] [Full Text]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
