Chest ACCP Education Calendar
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
 QUICK SEARCH:   [advanced]


     

Guest Access | Sign In via User Name/Password
This Article
Right arrow Full Text (PDF) Free
Right arrow Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Article Archive
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Joffe, A.
Right arrow Articles by Gould, M. K.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Joffe, A.
Right arrow Articles by Gould, M. K.
(Chest. 2005;127:411-412.)
© 2005 American College of Chest Physicians

Pitfalls in the Interpretation of Multivariable Models in the Critical Care Literature

Aaron Joffe, DO

St. Francis Medical Center, Honolulu, HI

Correspondence to: Aaron Joffe, DO, Staff Intensivist, St. Francis Medical Center, 2230 Liliha St, Honolulu, HI 96817; e-mail: doctrj{at}pol.net

To the Editor:

Recently, Barr and colleagues (April 2004)1 evaluated the effects of implementing a nutrition support protocol on the use of enteral nutrition, the time to the initiation of feedings, and the percentage of prescribed calories delivered. Duration of mechanical ventilation, ICU length of stay, hospital length of stay, and in-hospital mortality were studied as secondary end points. Using multivariable techniques, the authors concluded that their protocol increased the likelihood of receiving enteral nutrition and shortened the duration of mechanical ventilation, and that enteral nutrition reduced the risk of death. These conclusions depend on the integrity of the statistical analysis and, therefore, must be interpreted with caution.

Insofar as the ability of any model to predict or explain the studied outcome rests largely on the adjustment for variables that have been hypothesized on theoretic grounds or have been shown in previous research to be confounders of the relationship being studied,2 the inclusion of age, gender, severity of illness, admitting diagnosis, and nutritional status alone hardly represents a comprehensive list of variables that have previously been shown to affect the primary and secondary outcomes of this study. The use of prokinetic drugs such as metoclopramide and erythromycin may facilitate feeding tube placement and improve gastric emptying, thereby improving tolerance to enteral feeding, decreasing the time to the initiation of enteral feedings, and increasing the percentage of prescribed calories delivered,3 while ventilatory strategies limiting plateau pressures and employing lower tidal volumes have been shown to decrease mortality and to increase the number of ventilator-free days.4 Furthermore, the reliability of the Cox proportional hazards model showing a 56% lower risk of death in those receiving enteral nutrition contains an insufficient sample size. As a general rule, there must be at least 10 outcomes for each independent variable that is eligible to be included.5 Thus, 70 patients would have to have died in order to accommodate the seven variables included.

Last, the study suffers from omissions in reporting on factors that that have been previously shown to make proper and comprehensive interpretation of published pulmonary and critical care literature difficult, if not impossible (ie, the name of the statistical package used, testing results for interaction or effect modification, the testing result for collinearity between independent variables, specifying the reason for choosing the variables included in the model, and goodness-of-fit testing).6

In summary, Barr and colleagues1 have shown that the implementation of a nutritional management protocol in the ICU is feasible. Unfortunately, deficiencies in the manner in which the authors express the multivariate analyses make any further conclusions regarding the effects of a nutrition support protocol difficult and underscore the need for acceptable guidelines or criteria for the reporting of multivariable analyses in order to allow readers to more accurately interpret the results of future studies.

References

  1. Barr, J, Hecht, M, Flavin, K, et al (2004) Outcomes in critically ill patients before and after the implementation of an evidence-based nutritional management protocol. Chest 125,1446-1457[Abstract/Free Full Text]
  2. Katz, M Multivariable analysis: a primer for readers of medical research. Ann Intern Med 2003;138,644-650[Abstract/Free Full Text]
  3. Booth, C, Heyland, C, Patterson, W Gastrointestinal promotility drugs in the critical care setting: a systematic review of the evidence. Crit Care Med 2002;30,1429-1435[CrossRef][ISI][Medline]
  4. The Acute Respiratory Distress Syndrome Network. Ventilation with lower tidal volumes as compared with traditional tidal volumes for acute lung injury and the acute respiratory distress syndrome. N Engl J Med 2000;342,1301-1308[Abstract/Free Full Text]
  5. Concato, J, Feinstein, A, Holford, T The risk of determining risk with multivariable models. Ann Intern Med 1993;118,201-210[Abstract/Free Full Text]
  6. Moss, M, Wellman, D, Cotsonis, G An appraisal of multivariable logistic models in the pulmonary and critical care literature. Chest 2003;123,923-928[Abstract/Free Full Text]

Juliana Barr, MD and Michael K. Gould, MD, MS, FCCP

Veterans Affairs Palo Alto Health Care System, Palo Alto, CA Stanford University School of Medicine, Stanford, CA

Correspondence to: Julian Barr, MD, Veterans Affairs Palo Alto Health Care System, Stanford University School of Medicine, Anesthesiology Service (112A), VA Palo Health Care System, 3801 Miranda Avenue, Palo Alto, CA 94304; e-mail: barrj{at}stanford.edu

To the Editor:

We thank Dr. Joffe for his thoughtful comments regarding the use of multivariable models in critical care research that he wrote in response to our article that was recently published in CHEST (April 2004).1

Dr. Joffe expressed concern that we did not include several potential confounding variables in our multivariable models. While we agree that it is desirable to adjust for secular trends in management in a nonrandomized, interventional, before-and-after comparison such as ours, we limited the number of prespecified covariates in our models to avoid overfitting by using the rule of thumb that he describes in his letter (ie, at least 10 outcomes for each independent variable). Local practices for caring for critically ill patients, including ventilator management, were similar during the preimplementation and postimplementation phases of our study.

We disagree with Dr. Joffe’s comment about sample size. By definition, our study had sufficient power to demonstrate a statistically significant reduction in the risk of death in patients who were fed enterally.

At Dr. Joffe’s request, we are pleased to provide additional details about our methods and results. We selected covariates based on clinical judgment and entered all covariates into models without employing forward or backward selection procedures. We used Statview software (version 5.0.1; SAS Institute, Cary, NC) to perform all statistical analyses. Goodness-of-fit testing with the likelihood ratio and Pearson {chi}2 tests revealed that the model of enteral nutrition accounted for the outcome better than chance alone (p < 0.001), and that the predicted likelihood of the outcome was similar to the observed likelihood (p = 0.24). The results of the likelihood ratio and {chi}2 tests were similar for the model of mortality (p < 0.001 and p = 0.71, respectively). The correlation matrices of parameter estimates revealed no evidence of multicollinearity in models of enteral nutrition (R range, –0.14 to 0.23) or mortality (R range, –0.28 to 0.12). In post hoc analyses, we identified one statistically significant interaction between admission type and intervention group, such that the implementation of the nutritional support protocol had a greater effect on enteral feeding in surgical patients.

Based on this additional information, we hope that Dr. Joffe will agree that our conclusions are valid, namely, that the implementation of an evidence-based protocol for nutritional support resulted in a greater number of critically ill patients who were fed enterally and that enteral feeding was associated with improved survival.

References

  1. Barr, J, Hecht, M, Flavin, K, et al Outcomes in critically ill patients before and after the implementation of an evidence-based nutritional management protocol. Chest 2004;125,1446-1457




This Article
Right arrow Full Text (PDF) Free
Right arrow Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Article Archive
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Joffe, A.
Right arrow Articles by Gould, M. K.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Joffe, A.
Right arrow Articles by Gould, M. K.


HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS