Chest ACCP Member Benefits
HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS
QUICK SEARCH:   [advanced]


     

Guest Access | Sign In via User Name/Password
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF) Free
Right arrow Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Article Archive
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via ISI Web of Science (4)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Kahn, J. M.
Right arrow Articles by Rubenfeld, G. D.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Kahn, J. M.
Right arrow Articles by Rubenfeld, G. D.
(Chest. 2007;131:68-75.)
© 2007 American College of Chest Physicians

Transferring Critically Ill Patients Out of Hospital Improves the Standardized Mortality Ratio*

A Simulation Study

Jeremy M. Kahn, MD, MS; Andrew A. Kramer, PhD and Gordon D. Rubenfeld, MD, MSc

* From the Division of Pulmonary & Critical Care (Drs. Kahn and Rubenfeld), Harborview Medical Center, University of Washington, Seattle WA; and Cerner Corporation (Dr. Kramer), Kansas City, MO.

Correspondence to: Jeremy M. Kahn, MD, MS, Division of Pulmonary, Allergy, and Critical Care, University of Pennsylvania School of Medicine, 874 Maloney Building, 3600 Spruce St, Philadelphia, PA 19104; e-mail: jkahn{at}cceb.med.upenn.edu

Abstract

Background: Transferring critically ill patients to other acute care hospitals may artificially impact benchmarking measures. We sought to quantify the effect of out-of-hospital transfers on the standardized mortality ratio (SMR), an outcome-based measure of ICU performance.

Methods: We performed a cohort study and Monte Carlo simulation using data from 85 ICUs participating in the acute physiology and chronic health evaluation (APACHE) clinical information system from 2002 to 2003. The SMR (observed divided by expected hospital mortality) was calculated for each ICU using APACHE IV risk adjustment. A set number of patients was randomly assigned to be transferred out alive rather than experience their original outcome. The SMR was recalculated, and the mean simulated SMR was compared to the original.

Results: The mean (± SD) baseline SMR was 1.06 ± 0.19. In the simulation, increasing the number of transfers by 2% and 6% over baseline decreased the SMR by 0.10 ± 0.03 and 0.14 ± 0.03, respectively. At a 2% increase, 27 ICUs had a decrease in SMR of > 0.10, and two ICUs had a decrease in SMR of > 0.20. Transferring only one additional patient per month was enough to create a bias of > 0.1 in 27 ICUs.

Conclusions: Increasing the number of acute care transfers by a small amount can significantly bias the SMR, leading to incorrect inference about ICU quality. Sensitivity to the variation in hospital discharge practices greatly limits the use of the SMR as a quality measure.

Key Words: critical care • intensive care • Monte Carlo method • outcomes assessment • quality indicators

The movement toward increased accountability in medicine has prompted greater demand for valid and reliable measures of quality in the ICU.1234 One potential quality measure is the standardized mortality ratio (SMR), which is the observed hospital mortality divided by the expected mortality adjusting for severity of illness and case mix.5 An SMR of > 1 indicates higher than expected mortality, while an SMR of < 1 indicates lower than expected mortality. Theoretically, the SMR and other outcome-based quality measures can be used to compare outcomes across multiple ICUs or to follow temporal trends in a single ICU.678 Currently, the Joint Commission on Accreditation of Health Care Organizations (JCAHO) is considering the use of the SMR as a core measure of ICU quality in US hospitals.9

A major limitation of risk-adjusted outcome is that it can depend on factors other than the quality of care.10 In addition to true variation in quality, variation in the SMR can be due to poor reliability in data collection, random error, and variation in case mix.11 One additional factor may be patient discharge location. Many patients in the ICU are discharged directly to transitional care units such as skilled nursing facilities or long-term acute care centers (LTACs) or are transferred to regional medical centers for tertiary care. The outcomes of these patients are generally poor, with most studies reporting approximately a 50% in-hospital mortality rate for patients transferred to LTACs121314151617 and higher than expected mortality rate for those transferred to tertiary care centers.1819 ICUs that transfer a large number of patients may therefore have better than expected outcomes by shifting the mortality burden of high-risk patients.

Longitudinal data at the population level have shown that in years in which ICU transfers to nursing homes increase, the SMR tends to decrease.20 The effect of increasing out-of-hospital transfers at the individual ICU level, however, has not been directly examined. Furthermore, it is not known how many additional transfer patients are needed to produce a meaningful change in the SMR. To address this issue, we used cohort data from the acute physiology and chronic health evaluation (APACHE) IV clinical database to examine changes in the SMR at the ICU level after simulating an increase in the number of out-of-hospital transfers.

Materials and Methods

Study Design
We performed a cohort study and Monte Carlo simulation using data from the APACHE IV clinical database.21 APACHE is maintained by the Cerner Corporation (Kansas City, MO), which feeds back risk-adjusted outcome data to participating hospitals for benchmarking purposes. Detailed clinical information was prospectively collected on patients in 104 ICUs at 45 US hospitals during the calendar years 2002 and 2003. Data collection occurs on site by trained local coordinators. Although inconsistency in data collection procedures may occur, a three-phase standardized training session ensures that collection standards are uniform across sites, minimizing bias due to unreliable data collection techniques. All patients in the database were initially eligible for the study. Repeat hospital admissions and patients < 18 years of age were excluded from the study, as recommended in the JCAHO guidelines on ICU quality measures.9 Because it is difficult to accurately estimate risk-adjusted mortality when the number of patients is small, we also excluded ICUs with < 150 patients per year.

Variables and Analysis
Demographic and outcome data were summarized over each participating ICU. Original discharge location was determined by the ICU discharging nurse, and was coded as the hospital floor, telemetry, step-down unit, home, another ICU, another hospital (including skilled nursing facility or nursing home), or dead. Patients who were discharged to either another ICU or a hospital were grouped together as transfer patients for the baseline analysis. The specific location of transfer, such as nursing home, LTAC, or tertiary care center, was not available.

The SMR was calculated as the observed in-hospital mortality divided by the mean expected in-hospital mortality. The expected mortality rate for each patient was determined from the APACHE IV risk equation, which is a publicly available, validated model for predicting the outcome of critically ill hospitalized patients.21 The APACHE IV mortality equation has been recently updated and includes the day 1 acute physiology score, age, and select chronic health items (unchanged from APACHE III), as well as primary diagnosis, hospital admission location (including whether a patient was transferred in from an outside hospital), pre-ICU length of stay, and whether or not a sedated patient could have their Glasgow coma score assessed, whether or not a patient received ventilation during the first ICU day, and whether the patient had undergone emergency surgery. Confidence intervals (CIs) for the SMR of each ICU were obtained using bootstrap resampling, which is a nonparametric method that produces valid CIs accounting for the variability in both observed and expected mortality.22 To perform the bootstrap, the population of each ICU was sampled with replacement 1,000 times; the SMR was calculated for each bootstrapped sample using the observed mortality and the predicted mortality from the APACHE IV equation, and the non-bias-corrected percentiles were used as the 95% CIs.

Monte Carlo Simulation
The simulation was performed by randomly sampling a set percentage of patients to be transferred out of the ICU alive rather than experience their original outcome. A separate simulation was performed for each ICU. Increases of 2% and 6% above baseline were chosen to represent a range of effects, with transfer rates that were consistent with those in the data set and were similar to those for patients receiving prolonged ICU care in other cohorts.23 The sampling algorithm was based partly on the patient’s severity of illness, such that patients with a higher probability of death were more likely to be assigned for transfer (see "Appendix"). The algorithm was chosen so that the probability of hospital death for new transfers would approximate that of the mortality of patients who actually transferred out of the ICU.1213141516171819 An alternative strategy of sampling based on a propensity score was considered; however, many important factors related to transfer, such as bed availability at the accepting center, the goals of the treating physician, and patient and family preference, were unavailable. This made it unlikely that a propensity score could reliably identify patients who had a higher likelihood for transfer. We also considered a selection algorithm based on predicted ICU length of stay. In this cohort, however, patients were relatively equally likely to be transferred throughout their ICU stay, such that a length-of-stay algorithm would also not reliably identify patients who were at high likelihood for transfer.

Patients who were randomly selected for transfer then had their vital status at hospital discharge recoded to alive, and the SMR was recalculated based on the new observed mortality rate. The final transferred population therefore included all of those patients who actually transferred plus the 2% or 6% of patients randomly selected. The simulation was repeated 500 times with different random sets of assigned transfer patients (Fig 1 ). The mean probability of death and the mean new SMR from each simulation were recorded; transfer bias was calculated as the difference between the mean simulated SMR and the original SMR. The number of ICUs with bias outside the original 95% CI and the average number of patients transferred to create important bias were also calculated. Additionally, we performed a sensitivity analysis on the effect of transfer bias including the 19 ICUs that were originally excluded for low volume.


Figure 1
View larger version (15K):
[in this window]
[in a new window]
[Download PPT slide]
  		Figure 1. Schematic of the simulation. Large boxes indicate the entire ICU population; ovals indicate the actual and simulated transfer patients. In each simulation (SMR1), the SMR was recalculated with all transfer patients coded to survive (SMR2) and was repeated 500 times (SMR500) for each ICU. SMRbiased = outcome SMR of SMR1, SMR2, and SMR500.

 
Statistical analyses and simulations were preformed with a statistical software package (Stata, version 8.0; Stata Corp; College Station, TX). This research was approved by the University of Washington institutional review board.

Results

The complete cohort contained information on 131,618 admissions to 104 ICUs in 45 hospitals. We excluded 9,120 non-first admissions and 588 patients < 18 years of age. Nineteen ICUs in seven hospitals were excluded due to a volume of < 150 admissions per year (1,435 patients). The final cohort contained 120,475 patients in 85 ICUs. Unit demographics are shown Table 1 . There was significant variability in ICU type, academic status, size, and risk-adjusted outcome. ICUs transferred a median of 3% of patients out of hospital at baseline (interquartile range, 1.9 to 5.0), although this ranged as high as 17.4%.


View this table:
[in this window]
[in a new window]

  		Table 1. ICU Demographics*

 
The results of the Monte Carlo simulation are shown in Figure 2 and Table 2 . The simulations for each ICU indicate that increasing the number of out-of-hospital transfers results in a significant downward bias in the SMR at both 2% and 6% above baseline. As expected, severity of illness and APACHE IV risk of death of transfer patients increased as the number of transfers increased; however, the risk of death for transfers was still relatively low compared to that cited in the literature.1213141516171819 At an increase of 2% above baseline, nearly one third of ICUs experienced bias of > 0.1; at an increase of 6% above baseline, almost all ICUs experienced this degree of bias. In a large percentage of ICUs the new SMR was outside the original 95% CI, and many ICUs switched the direction of the quality indicator (ie, the SMR went from > 1.0 to < 1.0, such that a previously low-performing ICU appears to be high-performing). Figure 2 also shows that transfer bias can significantly affect ICU performance relative to each other (ie, rank). Were each ICU to be the only one to increase their number of transfers, they would significantly improve in rank compared to ICUs that did not increase their number of transfers. For example, the ICU with an original SMR of 1.27 (11th percentile in rank) would rise to the 28th percentile in the 2% simulation and the 37th percentile in the 6% simulation. The ICU with an original SMR of 1.05 (50th percentile in rank) would rise to the 67th percentile in the 2% simulation and the 75th percentile in the 6% simulation.


Figure 2
View larger version (18K):
[in this window]
[in a new window]
[Download PPT slide]
  		Figure 2. Change in SMR for the 85 ICUs in the Monte Carlo simulation, ranked according to their baseline SMR. Triangles and circles denote the mean SMR in simulations with an increased number of transfers of 2% and 6% above baseline, respectively. Lines indicate 95% CIs for each ICU obtained from bootstrap resampling.

 

View this table:
[in this window]
[in a new window]

  		Table 2. Bias in SMR With Increasing Numbers of Out-of-Hospital Transfers in 85 ICUs*

 
The mean (± SD) number of additional transfers per month was 1.2 ± 0.6 in the 2% simulation and 3.5 ± 1.9 in the 6% simulation. In the 2% simulation, 39 ICUs transferred only one additional patient per month or less, with an average bias of 0.10 ± 0.04. Of these, 16 still had bias > 0.1 despite the low numbers of patients. In the 6% simulation, 22 ICUs transferred ≤ 2% additional patients per month, with a mean bias of 0.16 ± 0.05. The bias was > 0.1 in all 22 of these ICUs.

Table 3 shows the mean transfer bias, stratified by the number of admissions, the baseline SMR, and the baseline percentage of transfer patients. Bias was marginally greater in smaller ICUs, low-performing ICUs, and ICUs with higher baseline transfer rates; however, in general the results were consistent across strata. Including the 19 small-volume ICUs in a sensitivity analysis did not appreciably change the size of the transfer bias in the simulation (mean transfer bias: 2% simulation, 0.09 ± 0.05; 6% simulation, 0.13 ± 0.05).


View this table:
[in this window]
[in a new window]

  		Table 3. Transfer Bias Stratified by ICU Characteristics*

 
Discussion

Several studies1819 have indicated that accepting critically ill transfer patients can adversely affect ICU benchmarking measures. This study documents that the reverse is true; that transferring even small numbers of critically ill patients out of the hospital can significantly improve the SMR of an ICU. In many ICUs, transferring only one additional patient per month was enough to cause significant bias. Bias could occur in ICUs of varying size, performance, and baseline number of transfer patients.

Transfer bias presents an important limitation to the SMR and other outcome-based quality measures. Although JCAHO is currently considering the use of the SMR as a core measure of hospital quality, a greater understanding of the factors affecting the SMR is needed before it should be widely used to benchmark ICU outcomes. Rather than reflecting differences in quality, the variation in the SMR could be explained in part by the variation in ICU discharge patterns. Transfer bias also introduces the possibility of "gaming" the system. Just as in some settings providers have avoided treating high-risk patients or up-coded the severity of illness to improve risk-adjusted outcome measures,242526 ICU providers could actively increase LTAC discharges to improve their SMR. Based on these results, ICU directors and accrediting organizations can monitor the number of transfer patients as a potential explanation for changes in the SMR over time. Transfer bias also has implications for pay-for-performance initiatives, in which hospitals receive bonus payments or other incentives based on the adoption of evidence-based care practices or improved risk-adjusted outcome. Pay for performance has long been advocated as a potential solution to the crisis in quality in US health care.27 Some evidence suggests, however, that pay for performance tends only to reward high-performing centers without motivating improvement in low-performing centers.28 This study shows that, were a pay-for-performance scheme to be based on the SMR, hospitals could be rewarded for changes in their discharge practices rather than actual changes in quality.

Many other limitations to the SMR exist. Current risk adjustment methodology may not entirely account for differences in the case mix across providers.1129 This is true even in the ICU, where detailed physiologic measurements are plentiful and statistical models for outcome prediction have been in use for decades.30 Different risk-stratification methods often give different results,313233 and even in the setting of detailed risk adjustment it is difficult to reliably identify quality outliers in the ICU.343536 Other threats to the validity of the SMR include random error and measurement error.

In light of these limitations, increased emphasis should be placed on process-based quality measures as a means of benchmarking ICU performance.7 Process measures can include rates of appropriate use of evidence-based therapies such as deep vein thrombosis prophylaxis, elevation of the head of the bed to prevent ventilator-associated pneumonia, and use of weaning and sedation protocols.37 Process-based measures have the advantage of offering clear strategies for improvement when quality is low and being less influenced by variations in case mix than outcome-based measures.38 Although more research is needed into the reliability and validity of these indicators, measuring and improving the process of care may represent the best way to improve the overall care of the critically ill.39

As a simulation study, this analysis is limited in that it may not accurately represent real-world transfer decisions. We could not model the complex decision-making process behind hospital discharges, and we did not factor in whether patients were transferred to LTACs or to tertiary care referral centers. Nonetheless, we based our simulation on a large multicenter database with careful risk adjustment, and have shown that the results were robust to baseline SMR, baseline transfer rate, and hospital size. Additionally, using Monte Carlo methodology introduces random variation into our estimates, such that the results are asymptotically true as different random groups of patients are selected for transfer.

The results of this analysis are also dependent on the algorithm used to randomly select patients for transfer. We chose a simple, easily understandable algorithm that resulted in newly assigned transfer patients having similar mortality rates to those reported in the literature. The resulting algorithm was actually conservative, as new transfer patients still had a predicted mortality rate of approximately 25%. Patients transferred to LTACs have reported mortality rates of 24 to 67%, with most studies reporting a > 50% mortality rate.121314151617 Patients transferred to tertiary care centers have reported mortality rates of 32 to 34%.1819 Thus, in practice the bias might be expected to be even larger than that we observed in this simulation. An alternative strategy that is not dependent on a simulation algorithm would have been an empiric study comparing the SMR among hospitals with varying levels of transfer or at a single hospital over time. There is no way to fully control for the multiple confounders related to transfer in such a study, including patient and physician preferences, making valid results unlikely.

This study shows that increasing the number of out-of-hospital transfers can bias the SMR, even when the number of transfer patients is small. Given the many sources of error in measurement and interpretation, caution is warranted in using the SMR as a benchmarking measure. Uncritical use of the SMR to benchmark ICU performance is likely to misinform rather than provide meaningful information about ICU quality. ICU directors and administrators can monitor the number of transfer patients as a potential explanation for a decreasing SMR.

Appendix

Details of Selection Algorithm
In each iteration of the simulation, transfer patients included those who were actually transferred from the ICU as well as those randomly selected by the simulation algorithm. The selection algorithm was designed to generate a random sample of patients for simulated transfer to an outside institution with an observed mortality rate similar to that of typical transfer patients, but independent of whether the patient actually died. Since patients are frequently transferred for a higher level of care, we wanted the probability of transfer to be linked to the patient’s severity of illness as measured by their APACHE IV score for risk of death.

For each simulation, patients who were not actually transferred out of the hospital were assigned a random number (x) with a uniform distribution between 0 and 1. The random number was added to the product of their APACHE IV risk of hospital death and a scaling constant of 0.2 to select patients with an observed mortality rate of approximately 20%. This final number was between 0 (for individuals with a random number of 0 and a predicted mortality rate of 0) and 1.2 (for individuals with a random number of 0.99 and a predicted mortality rate of 0.99). Patients were ranked by this number, and the upper two or six percentiles (p) were selected for transfer.

This algorithm ensured that every patient had a possibility of transfer but that sicker patients were more likely to be transferred by a factor based on severity of illness. The predicted mortality rate of transfer patients was between 0.20 and 0.30 for each simulation, as shown in Table 2.

Footnotes

Abbreviations: APACHE = acute physiology and chronic health evaluation; CI = confidence interval; JCAHO = Joint Commission on Accreditation of Healthcare Organizations; LTAC = long-term acute care center; SMR = standardized mortality ratio

Dr. Kahn has no financial conflicts of interest to disclose. Dr. Kramer is an employee of the Cerner Corporation and owns shares of Cerner Corporation stock. Dr. Rubenfeld is a paid consultant for the Cerner Corporation in matters unrelated to the APACHE clinical information system.

Received for publication March 20, 2006. Accepted for publication August 8, 2006.

References

  1. Blumenthal, D (1996) Part 1. Quality of care: what is it? N Engl J Med 335,891-894[Free Full Text]
  2. Institute of Medicine.. Crossing the quality chasm: a new health system for the 21st century. 2001 National Academy Press. Washington, DC:
  3. Garland, A Improving the ICU: part 1. Chest 2005;127,2151-2164
  4. Curtis, JR, Cook, DJ, Wall, RJ, et al Intensive care unit quality improvement: a "how-to" guide for the interdisciplinary team. Crit Care Med 2006;34,211-218[CrossRef][ISI][Medline]
  5. Iezzoni, LI Risk adjustment for measuring health care outcomes 3rd ed. 2003 Health Administration Press. Chicago, IL:
  6. Knaus, WA, Wagner, DP, Zimmerman, JE, et al Variations in mortality and length of stay in intensive care units. Ann Intern Med 1993;118,753-761[Abstract/Free Full Text]
  7. Pronovost, PJ, Nolan, T, Zeger, S, et al How can clinicians measure safety and quality in acute care? Lancet 2004;363,1061-1067[CrossRef][ISI][Medline]
  8. Angus, DC, Black, N Improving care of the critically ill: institutional and health-care system approaches. Lancet 2004;363,1314-1320[CrossRef][ISI][Medline]
  9. Joint Commission on Accreditation of Healthcare Organizations.. Specifications manual for national hospital quality measures-ICU, version 1.0 2005 Joint Commission on Accreditation of Healthcare Organizations. Oakbrook Terrace, IL:
  10. Lilford, R, Mohammed, MA, Spiegelhalter, D, et al Use and misuse of process and outcome data in managing performance of acute medical care: avoiding institutional stigma. Lancet 2004;363,1147-1154[CrossRef][ISI][Medline]
  11. Glance, LG, Osler, T, Shinozaki, T Effect of varying the case mix on the standardized mortality ratio and W statistic: a simulation study. Chest 2000;117,1112-1117
  12. Carson, SS, Bach, PB, Brzozowski, L, et al Outcomes after long-term acute care: an analysis of 133 mechanically ventilated patients. Am J Respir Crit Care Med 1999;159,1568-1573[Abstract/Free Full Text]
  13. Seneff, MG, Wagner, D, Thompson, D, et al The impact of long-term acute-care facilities on the outcome and cost of care for patients undergoing prolonged mechanical ventilation. Crit Care Med 2000;28,342-350[CrossRef][ISI][Medline]
  14. Scheinhorn, DJ, Chao, DC, Stearn-Hassenpflug, M, et al Outcomes in post-ICU mechanical ventilation: a therapist-implemented weaning protocol. Chest 2001;119,236-242
  15. Schonhofer, B, Euteneuer, S, Nava, S, et al Survival of mechanically ventilated patients admitted to a specialised weaning centre. Intensive Care Med 2002;28,908-916[CrossRef][ISI][Medline]
  16. Nasraway, SA, Button, GJ, Rand, WM, et al Survivors of catastrophic illness: outcome after direct transfer from intensive care to extended care facilities. Crit Care Med 2000;28,19-25[CrossRef][ISI][Medline]
  17. Dematte D’Amico, JE, Donnelly, HK, Mutlu, GM, et al Risk assessment for inpatient survival in the long-term acute care setting after prolonged critical illness. Chest 2003;124,1039-1045
  18. Rosenberg, AL, Hofer, TP, Strachan, C, et al Accepting critically ill transfer patients: adverse effect on a referral center’s outcome and benchmark measures. Ann Intern Med 2003;138,882-890[Abstract/Free Full Text]
  19. Combes, A, Luyt, CE, Trouillet, JL, et al Adverse effect on a referral intensive care unit’s performance of accepting patients transferred from another intensive care unit. Crit Care Med 2005;33,705-710[CrossRef][ISI][Medline]
  20. Sirio, CA, Shepardson, LB, Rotondi, AJ, et al Community-wide assessment of intensive care outcomes using a physiologically based prognostic measure: implications for critical care delivery from Cleveland Health Quality Choice. Chest 1999;115,793-801
  21. Zimmerman, JE, Kramer, AA, McNair, DS, et al Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today’s critically ill patients. Crit Care Med 2006;34,1297-1310[CrossRef][ISI][Medline]
  22. Hosmer, DW, Lemeshow, S Confidence interval estimates of an index of quality performance based on logistic regression models. Stat Med 1995;14,2161-2172[ISI][Medline]
  23. Carson, SS, Bach, PB The epidemiology and costs of chronic critical illness. Crit Care Clin 2002;18,461-476[CrossRef][ISI][Medline]
  24. Green, J, Wintfeld, N Report cards on cardiac surgeons: assessing New York State’s approach. N Engl J Med 1995;332,1229-1232[Free Full Text]
  25. Hofer, TP, Hayward, RA, Greenfield, S, et al The unreliability of individual physician "report cards" for assessing the costs and quality of care of a chronic disease. JAMA 1999;281,2098-2105[Abstract/Free Full Text]
  26. Burack, JH, Impellizzeri, P, Homel, P, et al Public reporting of surgical mortality: a survey of New York State cardiothoracic surgeons. Ann Thorac Surg 1999;68,1195-1200[Abstract/Free Full Text]
  27. Rosenthal, MB, Fernandopulle, R, Song, HR, et al Paying for quality: providers’ incentives for quality improvement. Health Aff (Millwood) 2004;23,127-141[Abstract/Free Full Text]
  28. Rosenthal, MB, Frank, RG, Li, Z, et al Early experience with pay-for-performance: from concept to practice. JAMA 2005;294,1788-1793[Abstract/Free Full Text]
  29. Pappachan, JV, Millar, B, Bennett, ED, et al Comparison of outcome from intensive care admission after adjustment for case mix by the APACHE III prognostic system. Chest 1999;115,802-810
  30. Knaus, WA, Wagner, DP, Draper, EA, et al The APACHE III prognostic system: risk prediction of hospital mortality for critically ill hospitalized adults. Chest 1991;100,1619-1636[Medline]
  31. Iezzoni, LI The risks of risk adjustment. JAMA 1997;278,1600-1607[Abstract]
  32. Glance, LG, Osler, TM, Dick, A Rating the quality of intensive care units: is it a function of the intensive care unit scoring system? Crit Care Med 2002;30,1976-1982[CrossRef][ISI][Medline]
  33. Glance, LG, Osler, TM, Dick, AW Identifying quality outliers in a large, multiple-institution database by using customized versions of the Simplified Acute Physiology Score II and the Mortality Probability Model II0. Crit Care Med 2002;30,1995-2002[CrossRef][ISI][Medline]
  34. Thomas, JW, Hofer, TP Accuracy of risk-adjusted mortality rate as a measure of hospital quality of care. Med Care 1999;37,83-92[CrossRef][ISI][Medline]
  35. Baker, DW, Einstadter, D, Thomas, C, et al The effect of publicly reporting hospital performance on market share and risk-adjusted mortality at high-mortality hospitals. Med Care 2003;41,729-740[CrossRef][ISI][Medline]
  36. Sivak, ED, Rogers, MA Assessing quality of care using in-hospital mortality: does it yield informed choices? Chest 1999;115,613-614
  37. Pronovost, PJ, Berenholtz, SM, Ngo, K, et al Developing and pilot testing quality indicators in the intensive care unit. J Crit Care 2003;18,145-155[CrossRef][ISI][Medline]
  38. Mant, J Process versus outcome indicators in the assessment of quality of health care. Int J Qual Health Care 2001;13,475-480[Abstract/Free Full Text]
  39. Garland, A Improving the ICU: part 2. Chest 2005;127,2165-2179




This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF) Free
Right arrow Submit a response
Right arrow Alert me when this article is cited
Right arrow Alert me when eLetters are posted
Right arrow Alert me if a correction is posted
Right arrow Citation Map
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Article Archive
Right arrow Download to citation manager
Right arrow reprints & permissions
Citing Articles
Right arrow Citing Articles via ISI Web of Science (4)
Right arrow Citing Articles via Google Scholar
Google Scholar
Right arrow Articles by Kahn, J. M.
Right arrow Articles by Rubenfeld, G. D.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Kahn, J. M.
Right arrow Articles by Rubenfeld, G. D.

HOME HELP FEEDBACK SUBSCRIPTIONS ARCHIVE SEARCH TABLE OF CONTENTS