Installing and Using the Software
Interpreting AHRQ QI Results
Specific Codes and Indicators
Installing and Using the Software
- What are the software requirements?
The AHRQ Quality Indicators (QIs) software is designed to read hospital administrative discharge data that generally conforms to HCUP specifications, in which each hospitalization is reported on a single record. The AHRQ QI software is possible because many hospitals and health organizations collect data that have common data elements and common data values.
In the data source used with the AHRQ QI software, each record must conform to the specifications listed in the Data Elements and Coding Conventions in the Software Instructions for SAS and WinQI which are available on the AHRQ Quality Indicators website. With only a few exceptions, the same specifications apply to all four modules. They all require diagnosis codes and procedure codes in ICD-9-CM format, in which the codes are in character format with significant leading zeroes and trailing blanks. Other variables include sex, age, admission type, discharge status, and diagnostic related group (DRG). This last variable (DRG) generally requires a grouper program, available through Centers for Medicare and Medicaid Services (CMS) or a third-party vendor.
Once you get your data into the proper format, you will be able to calculate provider-based (i.e., a specific hospital) observed rates because the numerator and the denominator for these are both drawn from your input data. However, area-based rates are available using Census data from the county in which the patients in your hospital reside. If this population is not meaningful for you, you may want to create your own alternate population file for more relevant denominators, as follows:
The software uses state and county FIPS codes to link the counties in which patients reside to a population file provided with the software. If your client or patient base is drawn from a wider area and Census data are not relevant, you can construct an alternate population file, assuming it contains a comparable data structure and uses the same coding conventions. The file must contain a record for each unique combination of state, county (FIPS codes), sex, age group (5 year bands), race, and population estimates for the years 1995 through 2014. While your alternate population file must contain these same data elements and coding conventions, you can use a unique identifier other than a FIPS state or county code to represent your alternate geographic entities, so long as the combination of these two codes can be matched to the PSTCO field in your input discharge data.
The Population File (POP95T14.TXT) must replicate the following format:
Field Variable Column Position Format Codes 1 State 1-2 Zero Filled Numeric FIPS Code 2 County 3-5 Zero Filled Numeric FIPS Code 3 Sex 7 Numeric 1=Male, 2=Female 4 Age Group 9-10 Numeric 1=0-4 years
5 Race 12 Numeric 1=White,
4=Asian & PI,
6 1995 Population 13-19 Numeric 7 1996 Population 20-26 Numeric 8 1997 Population 27-33 Numeric 9 1998 Population 34-40 Numeric 10 1999 Population 41-47 Numeric 11 2000 Population 48-54 Numeric 12 2001 Population 55-61 Numeric 13 2002 Population 62-68 Numeric 14 2003 Population 69-75 Numeric 15 2004 Population 76-82 Numeric 16 2005 Population 83-89 Numeric 17 2006 Population 90-96 Numeric 18 2007 Population 97-103 Numeric 19 2008 Population 104-110 Numeric 20 2009 Population 111-117 Numeric 21 2010 Population 118-124 Numeric 22 2011 Population 125-131 Numeric 23 2012 Population 132-138 Numeric 24 2013 Population 139-145 Numeric 25 2014 Population 146-152 Numeric
- What are the capabilities of the software?
The Quality Indicators Windows Application is designed to run as a single-user application, meaning two or more users are unable to share a database. The application is only available in a SAS QI® and WinQI version for a Microsoft Operating system. In general, the ease-of-use and case level analysis capabilities of the Windows software (WinQI) are geared toward the needs of hospitals and the open-source flexibility of the SAS software (SAS QI) is geared toward researchers.
AHRQ does not recommend trying to modify the programming code because of its complexity and the fact that the stratification logic is embedded throughout the program. If the user decides to modify the programming code, AHRQ cannot provide support on the modified program.
- How do I install the software?
- How do I upload my data?
- Will my data work with the program?
The software expects that the DRG or ICD-9 code on any given discharge record is valid for the fiscal year of the discharge date. The software is designed to be backwards compatible with previous fiscal year versions. For the status of the AHRQ QI software conversion to ICD-10 codes, see the Using the AHRQ QI Software section.
The AHRQ QI software ONLY accepts three common data formats: Text (comma separated values), Microsoft Access® and Microsoft Excel®. Please take note of the following key formatting issues:
- Each row of data represents a separate discharge record
- Each column of data represents a single variable for all discharges.
- Can I use less than one year’s worth of data?
Yes, you can report one quarter of data for provider rates. The only caution in doing this is that the relatively low frequency of events means that with the shorter time interval the rates may fluctuate from quarter to quarter -- more so than when reporting annual rates. This fluctuation, however, will be accounted for in the confidence interval -- in other words, the CI for a quarter of data is wider than the CI for a year of data. Bear in mind that, for area rates, given the use of Census data for the denominators, which assume one year of data, it is necessary to perform a proportional adjustment.
- Can I upload multiple data files?
You may only upload one file at a time. The previously uploaded file will be replaced by the new version.
- How do I generate reports?
In order to print a detailed report, set the macro %LET PRINT = 1 in the control file by looking for a banner marked “indicate if records should be printed at the end of each program.”
When you select either the area rates or the provider rates, you will see a succession of menus to guide you through all of the necessary selections and options. Both have menus to select the indicator, select date ranges (optional), select stratifiers (optional), and additional options such as risk-adjusted and smoothed rates. Provider indicators also have a menu to select hospitals (optional) and composite rates (optional). If you select a single stratifier (e.g., county for area rates or hospital for provider rates), the observed denominator will be in the fourth column from the left, following the indicator (column 1), the hospital (column 2), and the observed numerator (column 3). Of course, if you stratify on two variables, for example hospital and gender, then these values will fill the second and third column, with the observed numerator and denominator appearing in the fourth and fifth columns, respectively. Note that the denominator will apply to that particular combination of stratifier values, with an overall total just before the next indicator begins.
- Can I generate individual or custom reports?
The AHRQ Quality Indicators are risk-adjusted by specific variables such as age, gender, age and gender interaction (PQI, IQI, PSI and PDI), APR-DRG (IQI Only), DRG (PSI and PDI only), comorbidities (PSI and PDI only) and severity (IQI only). You can, however, stratify the risk-adjusted rates by variables such as hospital, metro area or by county, age categories, gender, race and pay category. You could stratify by another variable such as physician identifier with the Windows application (WinQI) by mapping this variable to one of the custom stratifiers and then selecting it in the Provider Report Wizard strata screen. The user may be able to do this with the SAS software (SAS QI) by treating (renaming) your physician identifier as the hospital identifier, but only for provider rates if this option is available in the data and provides useful information. There is likely to be more bias due to unobserved patient characteristics at the physician level, and the physician-level rates will be less reliable (i.e., will have more statistical noise).
- Can I transfer my files to another computer?
Yes. By saving your mapping (*.qim) file, you will be able to easily point WinQI to the same mapping file during the data load of your original data when recalling the original data within WinQI on a separate computer.
Updated April 20, 2015
- Common WinQI software issues
A few common reasons that the WinQI software may not work correctly are:
- Did you check the SQL server?
- If your disk volume is compressed the SQL server will not work properly. Use Windows® Internet Explorer® to look in Program Files/Microsoft® SQL Server®/AHRQQI Data to see if any files are blue instead of black. If so, follow the directions found in the WinQI software documentation.
- Have you changed your computer's network domain since you installed it?
- The database server name should be the name of that server/computer, so please try changing the local name to the actual server name via the program options in the AHRQ Quality Indicators software. If this doesn't work, then you will need to reinstall the application. During the database install, specify the actual server/computer name for the database server.
- Did you format the input data correctly?
- The most frequently cited Quality Indicators software error is due to the user not following the required data format exactly. For guidance on data format, please see the WinQI software instructions.
- When installing the application, did you install as admin?
- When you are attempting to open the downloaded .exe file (quality_indicators_with_apr46_x***.exe), instead of double clicking on the file name, right click and hit “run as administrator.”
- When opening the application, did you launch as admin?
- After installing the software, instead of double clicking on the file name, right click and select, “run as administrator.”
- Does this dialogue box indicate a problem during installation?
- The screen shot below provides an example of the kind of dialog box that users might see during installation of the AHRQ QI software.
This dialog box does not indicate a problem with the installation. When it appears, click the “Close” button and the installation will continue. Different components of the AHRQ QI software target different versions of the Visual C++ Redistributable Library. The dialog box simply notifies the user that an additional version of the library is being installed. This will not cause a problem with the execution of the AHRQ QI software or any other applications installed on the user’s computer.
- The screen shot below provides an example of the kind of dialog box that users might see during installation of the AHRQ QI software.
- Did you check the SQL server?
- Common SAS QI software issues
A few common reasons that the SAS QI software may not work correctly are:
- Are you using SAS Enterprise Guide?
- The SAS QI software is not intended for use with the Enterprise Guide application because certain elements of the code utilize the “X” command, which does not work in Enterprise Guide. The user must have access to Base SAS (including SAS/STAT and SAS/ACCESS) in order to run the SAS QI software.
- Did you properly modify the default path names and settings in the SAS programs?
- Comments are included in the CONTROL programs for each module to indicate which lines must be modified by the user. All references to “C:\Pathname\” in the SAS programs must be replaced with valid system paths to folders containing the referenced files. Once the CONTROL program has been modified, the path names must be changed in all other SAS programs within the same module to properly reference the location of the CONTROL program.
- Did you run the SAS programs in the proper order?
- Please consult the SAS Software Instructions or the comments near the top of each CONTROL program to determine the proper run order for the SAS programs in each module. If you intend to use APR-DRG values in the IQI module, the Limited License APR-DRG Grouper must be run before running any programs in the IQI module.
- Are you using the appropriate version of the Limited License APR-DRG Grouper (IQI only)?
- If you have SAS installed on a 32-bit Windows operating system (e.g., Windows XP), you need to download and run the 32-bit version of the Limited License APR-DRG Grouper.
- If you are using a 64-bit Windows operating system, you may be using 64-bit or 32-bit SAS, and the version of the Limited License APR-DRG Grouper must match your SAS installation. You can determine the version of SAS installed by selecting Start → All Programs → SAS → SAS 9.x.
- If you see “SAS 9.x (32) (English)” in the menu that is displayed, you have 32-bit SAS installed. You will need to download and run the 32-bit version of the Limited License APR-DRG Grouper.
- If you see “SAS 9.x (English)” in the menu that is displayed, you have 64-bit SAS installed. You will need to download and run the 64-bit version of the Limited License APR-DRG Grouper.
- Are you using the appropriate version of the Prediction Module software?
- Unlike the Limited License APR-DRG Grouper, the Prediction Module architecture must match that of your operating system, not your SAS installation. If you are using a 32-bit Windows operating system, you must download and install the 32-bit Prediction Module. If you have a 64-bit Windows operating system, you must install the 64-bit version of the Prediction Module.
*Note: This is only applicable to software versions prior to v5.0. In AHRQ QI v5.0, the Prediction Module was removed and there is no need for it to be installed for the SAS program to work.
- Are you using SAS Enterprise Guide?
Updated April 20, 2015
Interpreting AHRQ QI Results
- How do I interpret my reports?
The QI Toolkit can be used to help your hospital understand the AHRQ Quality Indicators for use in quality improvement and patient safety. The QI Toolkit section on Assessing Indicator Rates Using Trends and Benchmarks provides information on comparing and reporting on the AHRQ QI.
- What do the AHRQ QI rates mean and how are they calculated?
The following example illustrates the calculation and interpretation of AHRQ QI rates. An average provider rate of 0.001051, as provided by the software without the scale option, means that the average rate of hospitals with at least 1 case in the denominator was 1.05 per 1,000 or 0.1%. If the standard deviation on that average provider rate is 0.50 per 1,000, then approximately 2/3 of hospitals had rates between 0.55 and 1.60 per 1,000 (i.e., Average & Standard deviation) or rates between 0.055% and 0.16%. A population rate of 1.05 means that the average rate in the reference population (i.e., all discharges in the data file) was 1.05 per 1,000. Please note that the interpretation is based on how the rate is scaled (e.g., per 1,000 or 100,000).
Risk adjustment is highly specific to each QI. The indicators themselves are subject to in depth validation and expert panel review and are the products of an extensive process. They are designed to measure specific events (the numerator) for specific populations that are at risk (the denominator). Risk adjustment calculations and parameters used by the syntax are the product of a lengthy process that applies the syntax to a large national file of discharges and uses logistic regression analysis to calculate the risk-adjusting coefficients. The risk factors vary from indicator to indicator, as do the coefficients.
The expected rate and risk-adjusted rate are actually two separate concepts. A risk-adjusted rate is the rate the hospital would have if it had an average case mix. In other words, it holds the hospital's performance on the Quality Indicators constant and compares that to an average case mix. This is in contrast to an expected rate that holds the hospital's case mix constant and calculates the rate expected if the hospital performed at an average level. The expected rate is the rate that you would expect if your performance is the same as the national sample. It is the rate that the whole set of U.S. hospitals would perform if they all had the same demographics and case severity as your hospital.
- Expected value = discharge level outcome based on discharge level data (age, sex, etc.) calculated with a normative population experience
- Risk-Adjusted value = aggregate provider (hospital) value adjusted for a normative population experience (above) and hospital
- Why do my rates look low?
The expected rates are calculated from the appropriate coefficients for age group, gender, and other risk factors (e.g., the IQI uses APR-DRG, risk of mortality and severity group, while the PSI uses DRG group and comorbidity). The coefficients are calculated from hospital discharges collected from the Nationwide Inpatient Sample (NIS). Expected rates may be low because the sample is very different from the NIS sample or truncated in some way, or because the APR-DRG and other variables generated by the limited license grouper have not been included or generated correctly by the user.
- Can I use race in risk adjustment?
Race is only used as an optional stratification selected by the user. Race does not apply to any of the criteria used to define indicator numerators or denominators and is not used as a factor in risk adjustment. The AHRQ QI software only offers the option of stratifying AHRQ QI results according to the numeric values encountered in your data. If your codes do not conform to the specifications listed above, some output will be mislabeled. No calculations are affected other than stratifying results by the values in your data.
- Where do I learn about present on admission? (POA)
In data collected beginning October 1, 2007, each diagnosis code may be accompanied by a data element that indicates whether the diagnosed condition was Present-on-Admission (POA), and is therefore a pre-existing comorbidity, or whether the condition developed during the hospitalization of interest and is therefore a complication.
In prior versions of QI software prior to 5.0, a “prediction module” was used to impute missing POA information. Beginning with version 5.0 the QI software no longer uses the “prediction module”. The user of the AHRQ QI v5.0 software must specify whether or not the input data has POA information. Missing POA information is treated as if the condition is not present on admission.
For information about how POA was handled in earlier versions of the AHRQ QI software, refer to the resources section of this site and view the Webinar on Estimating Risk-Adjustment Models Incorporating Data on Present on Admission.
- How do smoothed rates and risk-adjusted rates differ?
Generally, the smoothed and risk-adjusted rates are very similar. If your interest is how your hospital or group of hospitals performs at a given time compared to a standardized case mix or standardized reference population, then you should use the risk-adjusted rate. If you are interested in how your facilities are most likely to do over time or in the future, you should rely on the smoothed rates.
- What are the formulas used?
Risk adjusted rates are calculated as the observed rate divided by the expected rate, times the population rate (O/E * P). The population rate is based on the entire population, not a sub-group, so when stratification is selected that confounds variables used in risk adjustment, the syntax presents only the O/E ratios.
- How do I calculate my rate per 1,000?
Calculate your observed rate by dividing your numerator by your denominator and multiplying the quotient by 1,000. AHRQ recommends using per 100,000 for counties or states; the state denominator will obviously be larger, but the numerator can be expected to be larger as well, so that both levels of measurement will use the same scale.
- How are outliers handled?
AHRQ provides the AHRQ QI software for users to use with their own hospital discharge data, so the responsibility for identifying outliers in the data lies with the user. Neither the AHRQ QIs, nor the software do this automatically. Additionally, there is no standard way to identify outliers when you are dealing with relatively rare events as many of the AHRQ QIs do. For a continuous variable measured among a large sample of records, you might indicate three or four standard deviations to constitute an outlier, but it is not really appropriate for rates.
- How are confidence intervals determined?
Because one is using information from a past time period to inform current decisions, the uncertainty in those decisions is reflected in the confidence intervals (CI). The CI for the risk adjusted rate is
Risk adjusted rate & standard error (SE) * 1.96
SE = (population mean/expected rate)*(1/population)*sqrt(expected rate variance).
The method used for the confidence intervals is David W. Hosmer, Stanly Lemeshow. Confidence interval estimates of an index of quality performance based on logistic regression models. Statistics in Medicine, Volume 14, Issue 19, pages 2161-2172 (October 1995).
The computation for the CI for the smoothed rate is located here.
- Do all of the AHRQ QI have confidence intervals?
AHRQ QIs reported as counts do not have confidence intervals (CI). Risk-adjusted rates of zero have confidence intervals because they are rounded to zero, while the observed rates are exactly zero and therefore don't have confidence intervals. The measures that are risk adjusted are included in the covariate tables linked here: PQI, IQI, PSI, PDI.
- How do I know if there is a statistically significant difference in the rates?
If the confidence intervals (CI) overlap, then there is no statistical difference; however, if they don't overlap, then there is significant statistical difference. If the CI is above the population rate then the outcome of interest is significantly higher than expected. However, if the CI is below the population rate then the outcome of interest is significantly lower than expected. It is up to you to determine if the statistical difference is clinically meaningful.
- How are the output data generated?
In SAS, each program's output file is used as input by the next program, e.g., PSSASP2.SAS and PSSASP3.SAS use the output file from the SAS program: PSSASP1.SAS. But the output files can also be used for special purposes by the user, since they contain the flag variables like TPPS06 that can be used for additional research or tabulations. Beginning in Version 4.3, the determination of whether a case is in the outcome of interest (TPPS06=1) or the population at risk (TPPS06=0) was complicated by the use of POA data in the P2 and P3 programs to exclude discharges that are present on admission. However, in general, if the indicator flag variable contains a 1, then the case is in the numerator. If the variable contains a 0, then the case is in the denominator but not in the numerator. For provider rates the denominator equals the number of valid records, or the number with either a 0 or a 1. If the variable contains the SAS missing code ("."), then the case was excluded from the indicator because the case did not meet the inclusion criteria or met one of the exclusion criteria.
To include all original variables plus flag variables in the output, the user must edit the "KEEP" phrase in the DATA statement in the P1 SAS program. Note that the record count in the output file will not be exactly equal to the record count in the input file because cases with missing age or sex (and some other data elements) are deleted. The record count will only be the same if these data elements are not missing.
In WinQI, all of the quality indicator logic is executed at once for AHRQ QI selected by the user and the rates are produced using the reporting wizard. The features of the reporting wizard are discussed in the WinQI software documentation.
- What is the difference between Area and Provider reports?
The Patient-level report displays results for a single record (single patient's discharge record) while the Area rate report (rates with Census data in the denominator) and the Provider rate report (rates with subsets of discharges in the denominator) present summary statistics on groups of discharges, depending upon the stratification you select.
- How do I determine the observed denominator and numerator (and observed rate) for PSI and PDI indicators that use the Markov Chain Monte Carlo (MCMC) Risk Adjustment Model?
When exporting data (specifically PSI and PDI discharge records) from WinQI versions 4.4 and 4.5 there are a few additional steps required to derive the observed numerator and denominator (and further, the observed rate) for a given indicator. To determine the Observed Numerator, please multiply the indicator value (PSI10) by the PSI10_wtdNum value and sum these by any strata of interest (e.g. Hospital ID). For the Observed Denominator, sum the values in the PSI10_weight field. These values can then be used to generate the Observed Rate by dividing the Observed Numerator / Observed Denominator (i.e. sum of (PSI10*PSI10_wtdNum) / sum of (PSI10_weight)).
SAS QI and WinQI v5.0 do not use the Markov Chain Monte Carlo (MCMC) Risk Adjustment Model. Indicators are risk adjusted using PROC SCORE in SAS with coefficients from the risk-adjustment models estimated using GEE or LOGISTIC models
For the SAS and WinQI prior to v5.0, not all PSI and PDI indicators are affected by this. The following PDI and PSI indicators are those that you should keep in mind when interpreting WinQI’s discharge output: NQI1, NQI2, NQI3, PDI1, PDI2, PDI5, PDI6, PDI8, PDI9, PDI10, PDI11, PDI12, PSI2, PSI3, PSI4, PSI6, PSI7, PSI8, PSI9, PSI10, PSI11, PSI12, PSI13, PSI14, and PSI15.
These indicators are also listed in table B1, Appendix B of the document titled Estimating Risk-Adjustment Models Incorporating Data on Present on Admission. Those indicators with an “X” in the “Measure Specifications” column use POA in their technical specifications or during flagging/exclusion.
Updated April 20, 2015
Specific Codes and Indicators
- AHRQ withdraws PSI 04 from NQF in 2017
PSI 04 or Death Rate Among Surgical Inpatients with Serious Treatable Complications was submitted to the National Quality Forum (NQF) for continued endorsement. After three rounds of intensive review at both the NQF Surgery Standing Committee and the NQF Consensus Standards Approval Process (CSAC), AHRQ withdrew the measure from further consideration at NQF. AHRQ conducted rigorous testing which demonstrated that the measure is valid and reliable.
Our findings were included in the materials submitted and reviewed at NQF (http://www.qualityforum.org/Publications/2017/04/Surgery_2015-2017_Final_Report.aspx). However, AHRQ has chosen not to continue with the NQF review process, pending a review of competing priorities. As with any measure withdrawn from consideration at NQF, endorsement was removed from the measure.
- Removal and replacement plan of PSI v6.0 ICD-9-CM SAS QI and WinQI software packages.
- PSI 90 Fact Sheet - Patient Safety and Adverse Events Composite “modified version” of PSI 90 for ICD-9-CM/PC for version 6.0.
- Potential Changes to PSI 90 in v6.0
Over the past year, the AHRQ Quality Indicators team has been testing a variety of approaches to improve the reliability, validity, and usefulness of PSI 90. These approaches include:
(1) adding more Patient Safety Indicators to the composite;
(2) allowing users to zero-weight the PSI 07 component, if they prefer to rely on central line associated bloodstream infection data reported through the National Healthcare Safety Network;
(3) modifying components of the composite to reduce their sensitivity to variation in documentation and coding practices across hospitals; and
(4) weighting component measures based not just on their relative frequency and reliability at the hospital level, but also on their relative severity or impact on population health. These approaches are under review at the National Quality Forum as part of the endorsement maintenance process (see http://www.qualityforum.org/QPS/0531 ; then click on “View the New Specification”).
Version 6 of the AHRQ Patient Safety Indicators software, scheduled for release in Spring of 2016, is expected to incorporate the changes specified above, such that the software will be flexible regarding the inclusion of PSI 07.
Added November 2, 2015
- Codes: How are the AHRQ Quality Indicator codes determined?
Overall, administrative data have the advantage of being populated by professional coders that use a common set of practices and guidelines, which brings some uniformity to the data that may be lacking in clinical data abstracted from medical records or recorded in electronic medical records (EMR) systems.
The AHRQ Quality Indicators (QIs) software compiles hospital inpatient administrative data that provides demographics on the patient and the provider, diagnosis codes, procedure codes and information about the admission, payer and discharge. The AHRQ QI software has been maintained to be backwards compatible and validly handle ICD-9-CM diagnosis and procedure codes in effect from 1994.
AHRQ QIs are a constant work in progress. They are continually being revised in response to new research or validation efforts, National Quality Forum (NQF) recommendations, or user feedback. If you have a question regarding a coding change, then consult the change logs for the relevant Quality Indicators (PQI, IQI, PSI, PDI). The change logs document all coding changes that occur. Also consider the technical specifications for each QI and examine which cases match each denominator to determine why each case was flagged.
- Codes: How are the codes used in the AHRQ QIs?
The denominator exclusion includes all of the codes in the numerator definition. For example, if any of the codes used in the numerator are in the principal diagnosis field, then the case is excluded from the denominator. Similarly, if any of the codes used in the numerator are in a secondary diagnosis field and are present on admission, then the case is excluded from the denominator. A patient meeting criteria for multiple measures will be included in each measure's denominator.
- In order to qualify as a coded condition, an incident must be:
- A condition or event that is not an EXPECTED or an inherent part of the procedure
- Clinically significant as determined by increased morbidity or additional care (including observation and diagnostic tests)
Codes are listed explicitly and do not imply that additional digits are included. For example, the PQI denote diagnosis codes as 3 or 4 digits, so codes with 5 digits are not accepted. The SAS QI formats have the definitive list of codes if there is some question about a particular code.
Any indicator that uses a population denominator (from U.S. Census) should use the patient FIPS code. Otherwise there might be cases in the numerator that are not included in the denominator.
- In order to qualify as a coded condition, an incident must be:
- Codes: Do the AHRQ QIs use E-codes?
Yes. A few of the indicators use E-code in the numerator, denominator or exclusion specifications. E-codes have different coding requirements than other ICD-9-CM codes, which require that a complication be coded only if it was unexpected and changed the course of care. Because national guidelines for E-codes do not require that a condition be an unexpected aspect of a procedure or disease in order to receive an additional code (i.e., an E-code), many minor and anticipated complications may be coded using these E-codes. Although several clinical panels have endorsed the concept of the indicator, in practice the types of cases identified are often not the type of complication originally envisioned by the panels during Quality Indicators measure development.
- Codes: Do the AHRQ QIs use CPT codes?
AHRQ does not support CPT (Current Procedural Terminology) codes for use with the AHRQ Quality Indicators that use inpatient administrative data.
- Composites: How do I interpret the composite rates?
The provider-level composite measure is the weighted average of the smoothed rates of a set of AHRQ QIs. The smoothed rate is a weighted average of the risk-adjusted rate and the reference population rate. Refer to the Empirical Method Document for details. The other rates are not part of the composite measure. Area-level composite measures, however, are calculated in the same manner as normal AHRQ QIs. The reliability-adjusted rates are the same as the smoothed rates. For additional information, see the composite user guides linked here: PQI, IQI, PSI, PDI. An increase in the composite rate is interpreted as an increase in the observed rate of the hospital relative to the expected rate. Examining each indicator can help determine what is driving the overall rate.
- DRG: How do I learn about MDC, DRG and groupers?
A crosswalk of the Major Diagnostic Category (MDC) and Diagnostic Related Group (DRG) is available here.
MDCs are used primarily in the inclusion rules for the covariate definitions. You should run a Center for Medicare and Medicaid Services (CMS) DRG grouper on your data to get DRG and MDC prior to running the AHRQ Quality Indicators (more detail is available on the software page for SAS or WinQI.
- The DRG used is the CMS DRG (Version 24 and before) or the MS-DRG (Version 25 and later) from the Centers for Medicare and Medicaid Services. There is no crosswalk between the APR-DRG and the CMS-DRG or MS-DRG. However, the software to calculate the CMS-DRG may be purchased from the National Technical Information Service.
- In most cases, MDCs are a straightforward aggregation of DRGs. For both SAS QI and WinQI, the software will make these aggregations automatically if the MDC is not already on the dataset. However, there are some DRGs that are not unique to an MDC; in these cases the MDC may only be assigned by the grouper.
- We use the 3M APR-DRGs (All Patient Refined DRGs) with the risk-of-mortality subclass in the risk adjustment for the IQI. We use a modified CMS DRG (collapsing with and without comorbid conditions or complications, or CC) and the AHRQ comorbidity software for the PSIs. We use the modified CMS DRG (collapsing with and without comorbid conditions or complications, or CC) or AHRQ CCS software for the PDIs. DRGs are not used in the PQIs.
- You cannot dummy code the DRG codes and get correct results out of the comorbidity software, because many of the comorbidities are defined in terms of DRGs.
- “Limited license grouper” means that not all of the APR-DRGs are included: only those that are actually used in the AHRQ Quality Indicators risk adjustment. So only about 30% of the discharges would have an APR-DRG and a risk-of-mortality subclass assigned. With the limited license grouper, the remaining discharges would have an APR-DRG value of zero. If you have your own APR-DRG grouper license, you can add the APR-DRG and risk-of-mortality data element to your input data file. 3M can offer technical support on how to use their grouper product. Once these data elements have been added to your input data file, you can use the AHRQ Quality Indicators software to load your data, and in the data element mapping step, map these data elements to your input data file data element. The software will then use the data elements from your input data file rather than using the limited license grouper.
Each year, DRG codes are updated-usually with new codes. The DRG version refers to the version in effect for a particular discharge record. The main purpose of DRG Version in the software is to distinguish CMS-DRG codes from Medicare Severity Diagnosis Related Group (MS-DRG) codes. Previous versions used discharge year/discharge quarter, where anything equal to or later than October 1, 2007 was considered MS-DRG. Not all hospitals had MS-DRG codes available after 2007/4 (i.e., discharge year/discharge quarter), so we changed the entry process to match HCUP where DRG versions 25-28 (as of October 2011) expect the DRG column data to be MS-DRG codes. Anything codes less than 25 (20-24) are considered CMS-DRG codes.
- DRG: How does the Low-Mortality DRG indicator work?
Conceptually, the rationale for the Low-Mortality DRG indicator is that the deaths that are flagged belong to DRGs with very low mortality rates (i.e., less than 0.5%). In other words, death is not an expected outcome for these DRGs. Cases that are flagged are considered never events (events that should not occur) like transfusion reaction and foreign body left in during a surgical procedure. Therefore, patients who belong to one DRG do not have a higher risk of death than patients in another DRG, and one might argue that risk adjustment is not appropriate. However, in the real world the lines between never events and potentially preventable events are not so clear. The mortality rate for some DRGs is higher than for other DRGs. In addition, users have expressed a preference for risk-adjustment and risk-adjustment was a requirement for NQF endorsement, so the indicator is risk-adjusted.
- PQI: Why are PQI #02 and PQI# 09 excluded from the composite?
Both of these indicators have discharge based denominators, rather than population denominators.
- PSI: How does PSI #12 Postoperative Pulmonary Embolism or Deep Vein Thrombosis Rate distinguish preoperative versus postoperative?
AHRQ is currently considering a proposal to rename the indicator PSI #12 Perioperative Pulmonary Embolism or Deep Vein Thrombosis Rate to reflect the inclusion in the numerator of both peri-operative and post-operative events. However, the current AHRQ PSI 12 logic cannot distinguish between preoperative (but hospital-acquired) and peri- or post-operative deep vein thromboses in surgical patients. In three separate validation studies on this indicator, 11%, 11% and 24% of the confirmed events initially presented before the index surgical procedure. Since the advent of "present on admission" coding and more specific ICD-9-CM codes for venous thromboses, these preoperative cases have become one of the leading causes of "false positive" cases of PSI 12. Of course, labeling of these cases as "false positive" is based on the current or proposed title of the indicator ("postoperative deep vein thrombosis..." or "peri-operative deep vein thrombosis...") but would not apply if the indicator were re-titled "hospital-acquired deep vein thrombosis..." as some have recommended.
The AHRQ QI team has considered, and will continue to consider, approaches to screen out these cases, such as excluding records in which the index surgical procedure is clearly delayed (as in the case described). The practical problem with these approaches is that they cannot distinguish cases in which the delay was under the control of the hospital and cases in which the delay was due to the natural progression of the patient's illness. For example, many patients admitted for hip fractures and other acute orthopedic injuries do not receive definitive surgery until four or more days into their hospital stay and they remain "uncovered" with thromboprophylaxis during this critical period. In this setting, a preoperative DVT may be considered a potentially preventable complication related to surgical delay, and would be appropriately captured by PSI 12.
- PSI: What logic was used in the development of PSI #4 Death Rate among Surgical Inpatients with Serious Treatable Complications?
The logic of PSI #4 is detailed in the AHRQ QI Development report; however, the measure focus is on the progression from complication to death (and the hospital’s ability to influence that progression); whether the hospital was responsible for causing the complication in this admission, a prior admission, or not at all is not material to that focus. Silber et al's more recent analysis (2007) also supports having as broad a denominator definition as possible, such as including conditions that were present on admission. In fact, Silber prefers an even broader denominator definition that would include every patient who dies in the hospital.
Absent any empirical evidence that restricting the denominator to conditions that arose during the same hospital stay would increase the validity of the indicator for assessing hospital quality, AHRQ has chosen to retain fidelity with the original concept of "failure to rescue", as it was developed by Silber et al. (1992) and adapted by Needleman et al. (2002). This rationale is explicitly discussed in Needleman's editorial (2007).
While the above discusses the rationale for the inclusion of conditions present on admission in the denominator, there is also a consideration of using the present on admission status in risk adjustment. At the present time, analysis and consideration is being given to the use of POA status as a covariate for a future version of the measure (that is, whether or not the condition that qualified the case for the denominator was POA).Some seminal articles influencing the development of this measure are:
- Silber JH, Williams SV, Krakauer H, Schwartz JS. Hospital and patient characteristics associated with death after surgery. A study of adverse occurrence and failure to rescue. Med Care 1992;30(7):615-29.
- Needleman J, Buerhaus P, Mattke S, Stewart M, Zelevinsky K. Nurse-staffing levels and the quality of care in hospitals. N Engl J Med 2002;346(22):1715-22.
- Needleman, J. & Buerhaus, P. (2007). Failure-to-Rescue: Comparing Definitions to Measure Quality of Care. Medical Care, 45(4).
- Silber, J., Romano, P., Rosen, A., Wang, Y., Even-Shoshan, O., & Volpp, K. (2007). Failure-to-Rescue Comparing Definitions to Measure Quality of Care. Medical Care, 45(4).
- Additionally supporting resources:
- Horwitz, L., Joanne, C., Cerese, J. and Krumholz, H. (2007). Failure to Rescue: Validation of an Algorithm Using Administrative Data, Medical Care, 45(4).
- McDonald, K., Davies, S., Geppert, J., & Romano, P. (2007). Why Rescue the Administrative Data Version of the “Failure to Rescue” Quality Indicator. Medical Care, 45(4).
- PDI: Why isn’t 518.5 included in Postoperative Respiratory Failure?
518.5 is a non-specific code, which includes traumatic respiratory failure, as well as respiratory distress, wet lung syndrome and idiopathic respiratory insufficiency for instance. We originally included this code in the software, but it was removed after chart review studies found it had a very high false positive rate.
However, we have since proposed changes to the ICD-9-CM system to increase the specificity of the codes and this proposal was discussed at the recent Coding and Maintenance Committee meeting. It is likely that this proposal or a very similar solution will be adopted and we will be able to further modify the indicator definition to capture more cases without sacrificing the specificity.
- Other: Do AHRQ QI consider ‘Do Not Resuscitate’ (DNR) orders?
The AHRQ QIs do not currently consider Do Not Resuscitate (DNR) orders as either a denominator exclusion or covariate in the risk-adjustment. AHRQ is, however, currently evaluating three relatively recent data elements related to hospice, palliative care and DNR. First, the UB-04 data element Point of Origin added a data value of “F” (Transfer from a Hospice Facility) in January 2010. Second, an ICD-9-CM diagnosis code V49.86 (Do not resuscitate status) was added October 1, 2010. Finally, the UB-04 data element Condition Code has a data value “P1” for “a DNR order was written at the time of or within the first 24 hours of the patient’s admission to the hospital and is clearly documented in the patient’s medical record”. The availability of HCUP data for 2010 will allow for empirical evaluation of one or more of these potential data elements alone or in combination as either an exclusion or covariate. As with any other potential patient characteristic, the empirical evaluation will focus on whether the characteristic is a mediator (and therefore a covariate) or moderator (and therefore a stratification or exclusion) of the quality of care.
- See AHA Coding Clinic for ICD-9-CM, 3Q 2008, Volume 25(32):13-14: "This code may be reported for any terminally ill patient who receives palliative care, regardless as to when the decision is made. There is no time limit or minimum for the use of this code assignment."
- See also AHA Coding Clinic for ICD-9-CM, 1Q 1998, Volume 15(1):11: "Terms such as comfort care, end-of-life care, and hospice care are all synonymous with palliative care. These, or similar terms, need to be written in the record to support the use of code V66.7."
In order to consider use of the V66.7 code, the coding guidance will need to be clarified or 5th digits must be included. AHRQ encourages professional societies with interest in this code to submit proposals to clarify the guidance and/or the creation of additional, more specific codes.
- Other: Will AHRQ create Quality Indicators for each type of surgery or specific condition?
The AHRQ Quality Indicators are focused on quality rather than prevalence and incidences of all cases. For example, the AHRQ QIs focus on the most acute strokes where evidence suggests that some of the variability among hospitals might be reduced through improved processes of care.
Creating Quality Indicators for specific conditions or types of surgery would result in rates that would likely be too narrow and based on too few cases to be of value. The indicators were developed and validated with clinical consultants, expert panels and considerable research as to their validity, reliability and usefulness in identifying classes of events that may be problems and that are actionable. Also, many are endorsed by the National Quality Forum (NQF).
There is a significant interest in additional stratifications of the data (e.g., hemorrhagic vs. ischemic stroke). Users of the Windows® software (WINQI) may use the custom stratification feature of the provider level reports to review risk-adjusted rates by these types of clinical classifications. One concern is reliability as you drill down into more specific strata because of the low frequency of many of the indicators.
Updated November 2, 2015