AHRQ on Provider Performance Reporting

Here is a great idea; lets give consumers and payers information on the quality and cost performance of physicians, hospitals and other providers, so that they can use that data to make good decisions about where to seek health care.  A simple idea, probably a very good one, but very difficult to implement from a data collection, analysis and reporting perspective.  And how useful is that information if it isn’t extremely trustworthy.  A recent report from the Agency for Healthcare Research and Quality examines in detail the methodological issues surrounding provider performance reporting, providing a very useful guide to providers themselves, payers and public policymakers.   (AHRQ Report)    The report was actually prepared under the auspices of the RAND think tank, and serves to highlight issues and possible solutions, but not to definitively recommend a particular approach.

The report addresses several key steps in the creation of the data to go into a performance report.  (Separate AHRQ papers deal with how to best design a report to ensure that the public can understand the data presented.)  Those steps include data aggregation, measure selection, data validation, attribution of data to providers, categorization of providers by level of performance and assessment of the likelihood of misclassifying a provider’s real performance.   The last area, misclassification, is dealt with first because of the serious consequences that flow from it.  Two types of misclassification exist, one is systemic and flows from use of performance measures which have low validity and the other is more statistical in nature, it is misclassification by chance, which can occur whenever you are using less than all the data available regarding a measure.

Measure selection is another complex area.  Ideally measures should be those that are directly linked to outcomes that matter to patients and payers, have a very strong relationship with those outcomes and for which data is easy to collect and widely available.  The patient populations, and the characteristics of those populations, can have a bearing on the results of a measure for a given provider, so measure results need to be able to be adjusted for those factors.   Different kinds of data may have more or less bearing on a measure, may be more or less expensive to gather and may have higher or lower levels of accuracy, so selecting data and data sources is important.  In many cases patients see multiple providers, so which one will have data relating to the patient attributed to it can be a critical problem.  As can be seen just from this brief description, there are a plethora of issues on performance reporting and the AHRQ work is almost invaluable in identifying and thinking about how to deal with them.