Exploring Hospital Resiliency by Examining AHRQ PSI-04 Rates Early in the Pandemic
Comparing performance for the patient safety indicator "Death Rate among Surgical Inpatients with Serious Treatable Complications (PSI-04)" during the early pandemic
The Agency for Healthcare Research and Quality's (AHRQ) Patient Safety Indicator 04 (PSI-04) is a well-known measure of hospital quality that calculates a hospital's rate of death following elective surgeries that result in treatable complications. Relevant to this research, the July 2022 Patient Safety Indicator 04 (PSI 04) measured the death rate among surgical inpatients with serious treatable complications.
The Science of Quality Measurement Program team has advocated for the importance of understanding hospital resiliency by reporting hospital quality outcomes during the early pandemic. The team defines the "early pandemic" as the period from April through Dec 2020, before vaccines or COVID-specific treatment options were available.
Analyzing PSI-04 rates offers a unique opportunity to understand early pandemic hospital resiliency. Elective surgeries were less directly impacted by COVID than other types of care because patients with active COVID infections were not routinely able to undergo elective surgeries. The team therefore hypothesizes that hospitals' PSI-04 performances should have been more stable during the early pandemic than many other patient safety outcomes — such as PSI-12, the perioperative pulmonary embolism or deep vein thrombosis rate — which were directly affected by patients' COVID-19 status.
The program team tested whether hospitals with better quality ratings before the pandemic were more resilient, in terms of PSI-04 rates during the early pandemic, than hospitals with lower quality ratings.
First, researchers split hospitals into quartiles, using the Centers for Medicare & Medicaid Services (CMS) Five-Star Quality Rating System, also called CMS Star Ratings — specifically, they used the Overall Hospital Quality Star Rating Hospital-Specific Report User Guide July 2022 publication. The top 25% of scorers formed the "Top Quartile," and all other hospitals formed the "Bottom 3 Quartiles." Then, the team plotted monthly PSI-04 rates, as a proportion, from January 2019 through December 2020, comparing the Top Quartile versus Bottom 3 Quartiles of hospitals.
Figure 1
PSI-04 rate from January 2019 to December 2020 by quartiles of CMS Star Rating
Figure 2
Difference in PSI-04 rate from January 2019 to December 2020, comparing Bottom 3 Quartiles with Top Quartile of CMS Star Rating
Among both groups, the team noted that PSI-04 rates were higher during the early pandemic compared with the period from January 2019 through March 2020. PSI-04 rates rose from 15.8% to 17.3% among Bottom 3 Quartile hospitals and from 13.1% to 14.1% among Top Quartile hospitals (Figure 1) during the early pandemic.
At first glance, this data suggests that pre-pandemic hospital quality processes and structures may not be related to early pandemic quality performance, considering that both top and bottom performers saw worse PSI-04 rates during this period. However, the team found an interesting pattern by comparing the difference in PSI-04 rates between Top Quartile and Bottom 3 Quartile hospitals (Figure 2).
The team noted that Top Quartile hospitals outperformed Bottom 3 Quartile hospitals by a greater margin during the early pandemic. Before the pandemic, Top Quartile hospitals had PSI-04 rates 2.7% better than Bottom 3 Quartile hospitals. This difference actually increased to 3.2% during the early pandemic. Notably, Bottom 3 Quartile hospitals had their highest PSI-04 rate — 17.7% — during April 2020, the first month of the early pandemic. But Top Quartile hospitals saw only a 13.2% PSI-04 rate in April 2020, which was nearly identical to their pre-pandemic performance.
The fact that the PSI-04 rate increased during the first month of the pandemic among Bottom 3 Quartile hospitals but not Top Quartile hospitals may support the hypothesis that there is some existing association between hospital quality and resiliency. More rigorous analyses could risk-adjust for hospitals' patient case mixes, COVID-19 census and comorbidity burdens.
Take-home point: Attempting to learn from hospitals that had the best quality performance despite the pandemic continues to be a worthwhile pursuit. Doing so may uncover interesting findings that could help hospitals better prepare for future pandemics or health system crises.
Related publications:
Contact