| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
Guest Access | Sign In via User Name/Password
|
|
|
|||||||
Guest Access | Sign In via User Name/Password |
|||||||||
* From the Department of Physical Therapy (Dr. Brooks, Ms. Sinclair, and Ms. Quirt), University of Toronto, Toronto, ON, Canada; the Department of Physical Therapy (Ms. Parsons) Mount Sinai Hospital, University Musculoskeletal Oncology Unit, Toronto, ON, Canada; the Department of Physical Therapy (Ms. Newton), St. Michael’s Hospital, Toronto, ON, Canada; the Department of Physical Therapy (Ms. Dear), University Health Network, Toronto, ON, Canada; and the Department of Physical Therapy (Ms. Silaj), Sunnybrook and Women’s College Health Sciences Center, Toronto, ON, Canada.
Correspondence to: Dina Brooks, PhD, MSc, BSc (PT), Department of Physical Therapy, 256 McCaul St, Toronto, ON M5T 1W5, Canada; e-mail: dina.brooks{at}utoronto.ca
 |
Abstract |
|---|
 TOP Abstract IntroductionMaterials and MethodsResultsDiscussionConclusionReferences |
|---|
Design: We developed the postoperative physiotherapy discharge scoring tool (POP-DST), a tool composed of objective criteria and a scoring system that would be used to determine when a patient should be discharged from perioperative PT. It is a composite score of the following five subcategories: mobility; breath sounds; secretion clearance; oxygen saturation; and respiratory rate. The score for the POP-DST ranges from 6 to 15, with a score of > 13 indicating readiness for discharge. We examined the content validity of the the POP-DST using focus groups and a mailed survey. To determine interrater reliability, two therapists, who were blinded to each other’s scores, assessed postsurgical patients. Validity was examined by comparing the decision to discharge based on the score on the POP-DST to the decision to discharge according to the therapist’s judgment. In addition, subjects who were discharged from PT were followed-up 7 to 10 days later to determine whether they had developed any subsequent respiratory problems.
Patients: One hundred four surgical patients were assessed to determine the reliability and validity of the POP-DST. For the ability of the test to detect postoperative complications following discharge from PT, 204 surgical patients were followed-up after discharge from PT.
Results: Interrater reliability was
moderately high (intraclass correlation coefficient = 0.76;
r = 0.77). There was strong agreement between the
decision to discharge the patient from PT based on the tool criteria
compared to the therapist’s judgment (
range, 0.91 to 0.96). The
ability of the POP-DST to predict those patients who would not develop
complications postoperatively was 94%.
Conclusion: The results indicate that the POP-DST would facilitate clinical decision making related to PT discharge planning in postsurgical populations. The instrument demonstrated strong content validity and predictive validity, as well as high levels of interobserver agreement. This tool should be considered as a work in progress until it is more fully validated.
Key Words: measurement properties • physical therapy • rehabilitation • surgery
|
Introduction |
|---|
|
TOPAbstractIntroductionMaterials and MethodsResultsDiscussionConclusionReferences |
|---|
A recently published clinical practice guideline6 on perioperative PT evaluated the evidence for the effectiveness of breathing exercises, incentive spirometry, manual techniques, and other techniques used in this patient population. None of the studies that examined the effectiveness of perioperative PT incorporated objective PT discharge criteria. Most of the studies applied postoperative PT treatment for a set number of days (range, 1 to 9 days)3 7 or for an unspecified length of time based on the judgment of the physical therapist.8
In 1996, an unpublished survey of Canadian physical therapists practicing in the cardiorespiratory field included a case scenario of a patient who had undergone cholecystectomy surgery. The respondents were asked to answer the following question: "what criteria do you use to discharge this patient from PT care?" The most frequent responses were independence in mobilization/ambulation, normal breath sounds, and an effective cough. Other responses included the following: the presence of a normal temperature; the return of outcome measures to baseline values; a normal chest radiograph; stable vital signs; the return of pulmonary function values to normal (including oxygen saturation by pulse oximetry [SpO2]); and normal mental status. This same question was asked of a similar sample of physical therapists in 1990, and the same criteria were identified at that time. Thus, there are no established valid and reliable criteria to be used when discharging patients from perioperative PT. Informal results show that a large number of criteria are presently used in the clinical setting and that these criteria have been consistently used over the last 8 years.
The objectives of this study were to undertake a multicentered study in order to accomplish the following: (1) to develop a tool, the postoperative physiotherapy discharge scoring tool (POP-DST), that would be used to assess patient readiness for discharge from postoperative PT care; (2) to establish the content and predictive validity of the POP-DST; and (3) to examine the interrater reliability of the POP-DST.
|
Materials and Methods |
|---|
|
TOPAbstractIntroductionMaterials and MethodsResultsDiscussionConclusionReferences |
|---|
Phase 1
Based on the available literature, we developed a draft of the
POP-DST (consisting of objective criteria and a scoring system) for
evaluating patient readiness for discharge from postoperative PT. The
criteria were multidimensional and were based on the results of
objective tests (eg, oximetry) and the findings on physical
examinations (eg, auscultation). Each item or criterion was
scored and weighted independently. A total score for the sum of all
items then was calculated. A given score was established as
representing the threshold for discharging patients from PT care (see
the final version given in Table 1
).
|
|
Two physical therapists assessed each patient. To minimize the effect of the change in patient status on the results, the assessments by the two therapists were performed on the same day (within a 4-h period). Each therapist was blinded to the other therapist’s findings.
The therapists were unaware of the threshold established as indicating readiness for PT discharge (ie, total score, > 13). Nevertheless, the therapists were asked the following question: "based on your clinical judgment, would you discharge this patient from physiotherapy?" To determine the validity of the threshold, we compared whether the total score established by the POP-DST (above or below the threshold value) was in agreement with the therapist’s global assessment.
To determine predictive validity, 204 patients were monitored for 7 to 10 days postdischarge from PT. After discharge from PT, we observed patients in the hospital or at home by telephone using a standardized form to determine whether they had developed pulmonary complications that could have been prevented or treated by PT. If the patient was still in hospital, we examined the chart to determine whether there were any pulmonary complications that had been documented and whether PT had been reinitiated. If the patient was discharged from the hospital to home, we inquired from the patient whether the patient had experienced any problems with breathing since discharge from the hospital and attempted to determine the diagnosis and intervention offered.
We hypothesized that if the POP-DST were valid and predictive, reaching its threshold value would be associated with a lower incidence of pulmonary complications after discharge from PT. However, given that other factors outside the realm of PT could result in the development of complications, an occurrence rate of > 10% was considered to be clinically important.
Subjects
In phase 1, the focus groups included a total of 37 clinical
physical therapists, 4 of whom were considered to be specialists in
cardiorespiratory care. Feedback was sought by a mailed survey from 23
other participants.
In phase 2, 104 patients were recruited prospectively and were assessed by two physical therapists. The patients had undergone cardiovascular surgery (n = 60), thoracic surgery (n = 13), abdominal surgery (n = 24), head and neck surgery (n = 3), or other surgery (n = 4). For the follow-up portion of the study, 204 patients were included, 132 of whom had undergone cardiovascular surgery, 39 of whom had undergone thoracic surgery, and 33 of whom had undergone major abdominal surgery.
For the phase 2 sample size calculation, we speculated that the difference in the discharge criteria scoring system that would correspond to a small but clinically important change was 1. The anticipated pooled SD was estimated at 2 (ie, the approximate range divided by 4).10 Using a two-tailed test, with type I error of 0.05 and a power of 90%, a significant difference would be detected with a minimum of 86 patients.
Statistical Analysis
Interrater reliability was determined by calculating the
intraclass correlation coefficient (ICC) and the Pearson correlation
coefficient (r value). A t test was used to
determine whether the difference between the scores of two therapists
was statistically significant. For r values, 0.90 to 1.0 is
considered to indicate high reliability, 0.80 to 0.89 to indicate good
reliability, 0.70 to 0.79 to indicate fair reliability, and < 0.69 to
indicate poor reliability. The ICC is a stronger measure for evaluating
interobserver reliability as it takes into account both the
between-subject and within-subject components of variance
(ie, between-rater and within-rater variability as well as
the heterogeneity of the sample).11
For ICC, values of
> 0.60 are considered to have acceptable reliability.12
In addition, we considered the agreement between the therapist’s
global assessment of readiness for discharge from PT and the decision
based on the POP-DST using Cohen’s statistic.11
13
14
The statistic measures agreement based on dichotomous variables.
values of > 0.75 are considered to be representative of excellent
reproducibility, while those ranging from 0.40 to 0.74 are thought to
reflect good reproducibility,14
depending on the type of
measure evaluated and the expertise of the raters.12
For the follow-up study, we determined the percentage of subjects who developed respiratory complications who were amenable to PT intervention following discharge from PT. We also calculated the positive and negative predictive values for the POP-DST. The positive predictive value is the probability of having a positive result on a test and having the condition of interest (in this case, post-PT discharge respiratory complications) relative to the total number of subjects scoring positive on the test (irrespective of whether they have the condition or not).15 The negative predictive value is the ratio of those patients who have a negative test result and do not have complications relative to the total number of subjects who have negative test results.15
|
Results |
|---|
|
TOPAbstractIntroductionMaterials and MethodsResultsDiscussionConclusionReferences |
|---|
Twenty of the 23 questionnaires were returned (return rate, 85%). Of the 20 respondents, 18 individuals completed the questionnaire and 2 wrote note/essay feedback and comments. Five respondents were physicians (including surgeons, respirologists, and anesthetists), eight were academic physical therapists, and seven were physical therapists who were considered to be specialists in the field of cardiorespiratory PT. Generally, support for the tool was high with 89 to 94% of respondents identifying the tool as beneficial and stating that they would encourage their colleagues to use it. Only four respondents identified additional variables to add to the tool, such as pain control, chest radiography, and patient cooperation. One respondent suggested removing RR from the tool. Overall, ambulation status was rated as the most important indicator in determining readiness for discharge from PT and RR was rated as the least important indicator, but the relative ratings differed based on professional designation. Physicians listed mobility as one of the least important variables and ranked RR as more important. In contrast, physical therapists ranked mobility as very important and RR as least important. Overall, RR and SpO2 were rated the lowest in terms of importance.
Based on all the feedback received, the tool was modified to the version seen in Table 1 . It is a composite score of the following five subcategories: mobility; breath sounds; secretion clearance; SpO2; and RR. The score for the tool ranges from 6 to 15, with a score of > 13 indicating readiness for discharge. The modifications did not alter the five subcategories, as these had been the original ones chosen. The wording was modified to improve clarity, and a resource sheet for the POP-DST was developed to outline definitions and guidelines. Based on the ranking of the subcategories, their weight was modified to reflect the importance placed on these variables (ie, lower weight for RR and saturation). Note that a higher number for an abnormal finding indicates less weight for that subcategory.
Reliability
Figure 1
shows the correlation between the score calculated by one therapist vs
that calculated by the other therapist. There was a strong correlation
between the two scores. Table 3 summarizes the correlation between the scores from the two therapists
for each of the subcategories. r values and ICCs were
performed and revealed similar results (r = 0.77; ICC,
0.76), indicating that the total score is highly reproducible. With
respect to the subscales, mobilization and pulse oximetry were the most
reliable, while RR was the least reliable. There was no effect of order
(repeated-measures analysis of variance for all subcategories,
p > 0.1). A paired t test revealed that there was
no significant difference between the overall scores from the two
therapists (p < 0.001).
|
|
statistics for agreement between the
therapists’ judgment and the POP-DST criteria ranged between 0.910 and
0.956. However, agreement between the two therapists (regardless of the
score on the POP-DST) was lower, with a of 0.690.
|
|
Discussion |
|---|
|
TOPAbstractIntroductionMaterials and MethodsResultsDiscussionConclusionReferences |
|---|
Although this tool can be used with any surgical population, it is most applicable to patients undergoing thoracic, cardiac, and upper abdominal surgical procedures. Given the high-risk nature of these surgical populations,16 coupled with the scarcity of PT and overall health-care resources, it is vital to identify those patients who are ready for discharge from postoperative care and those who are at risk for major morbidity. Patients still considered to be at risk (based on objective criteria as included in the POP-DST) should continue to receive PT interventions, while those who are no longer at risk should be discharged to permit the treatment of other patients. Thus, this tool will assist with caseload management.
Davis and colleagues17 reviewed seven standards of measurement for evaluating the usefulness of instruments in clinical practice. These standards include the following: the stated purpose of the measure; its conceptualization; its content (both breadth and depth); its feasibility; its cross-sectional and longitudinal reliability; as well as its validity (including responsiveness to change).17 The purpose of the POP-DST (and its intended populations) has been outlined. Our qualitative methodology in phase 1 provided the tool with content validity and detailed its conceptual framework. We have chosen its component criteria, its scoring, and the weighting of it items with care. However, we have not evaluated its floor and ceiling effects.2 3 These should be the focus for future studies. The interrater reliability of the POP-DST is acceptable, but we have not yet examined its test-retest reliability (ie, longitudinal reliability), and we have not formally evaluated its internal consistency (see below). Finally, we have taken steps toward establishing the validity of the POP-DST by assessing its construct validity (by comparing the tool with clinical judgment) and its predictive validity, but we have not been able to establish its criterion validity.
Phase 1 of our study evaluated the content validity of the draft tool and led to the development of the version tested in phase 2. We adopted a qualitative methodology, whereby feedback was sought regarding the criteria employed, the relative weighting of the items, and the scoring system. We used a multimodality approach for establishing content validity, namely, focus groups and questionnaires. Participants included physical therapists with a range of experience and expertise in the area who worked in varied settings (from large teaching hospitals and tertiary-care centers), from academic experts, and from physicians who were knowledgeable in postoperative care. This broad-based feedback allowed us to draft a tool that was clinically meaningful, feasible, and more methodologically rigorous.
The POP-DST total score demonstrated acceptable interrater reliability,
as did the mobility and pulse oximetry subscales. The high reliability
seen with pulse oximetry is not surprising given the technological
objective nature of this measure. Heavy weighting was accorded to the
mobility subscale, therefore it is encouraging that it demonstrated
such high reliability coefficients. Measures such as auscultation and
physical examination (including RR and sputum production) are more
subjective and therefore relatively unreliable. The lower reliability
coefficients found for the auscultation, secretion clearance, and RR
subscales in our study reflect the nature of these measures. While many
clinicians and researchers recognize their limitations, these three
measures continue to be widely employed, but not in isolation. It was
our intention that the POP-DST should reflect current clinical practice
in its reliance on multiple measures. No single measure has been
identified as a "gold standard" for cardiorespiratory care, making
a composite measure preferable. While excellent agreement was achieved
between therapists’ judgments and the tool threshold value (regarding
decisions to discharge), it is interesting that statistics were
lower for between-therapist agreement based on clinical judgment alone.
This suggests that the tool reflects clinical judgment regarding the
decision to discharge the patient from PT, but that the use of
objective criteria is more reliable than clinical impression alone. It
is likely that objective tools (such as the POP-DST) therefore will
enhance clinical decision making.
The instrument demonstrates predictive validity. The negative predictive value was 94.4%, meaning that most patients with a negative test result (ie, a total score above the threshold value of 13 points) did not experience postoperative pulmonary complications/symptomatology following discharge from PT care. The positive predictive value obtained was only 8.7%, which is hardly surprising given the low prevalence of postdischarge respiratory complications (particularly those amenable to PT). Streiner and colleagues15 have noted that the prevalence of a condition greatly affects predictive values. Where prevalence is low, positive predictive values tend to be low and negative predictive values tend to be high.15
This study had a number of limitations. First, we compared the threshold value to global assessment by therapists but employed no other objective measure in our investigation. However, there are few other measures that would be cost-effective and reliable for comparison purposes. Furthermore, as no "gold standard" for determining the readiness for discharge of a patient from PT exists in the literature, we were unable to establish criterion validity. With respect to evaluating predictive validity, we relied on a self-reported measure of complications, and we had no objective measure for confirmation. As have many other investigators, we have found the lack of individual strong cardiorespiratory outcome measures problematic.
While we attempted to design a tool that was feasible and applicable to a broad range of settings, an issue of feasibility was raised in the focus groups. Some facilities indicated that pulse oximetry is not routinely employed by their therapists, and this would make widespread use of the POP-DST problematic. Dropping this component of the POP-DST would not be advisable, given its high reliability. In contrast, the findings from this study may provide an argument for using the pulse oximeter in this patient population.
|
Conclusion |
|---|
|
TOPAbstractIntroductionMaterials and MethodsResultsDiscussionConclusionReferences |
|---|
|
Footnotes |
|---|
This research was supported by the Ontario Respiratory Care Society and the Canadian Physiotherapy Cardiorespiratory Society.
Received for publication April 25, 2001. Accepted for publication August 15, 2001.
|
References |
|---|
|
TOPAbstractIntroductionMaterials and MethodsResultsDiscussionConclusionReferences |
|---|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
