Evaluating Hospital Rating Systems

Okay, so if you believe that giving people good information about health care and health care providers will help them be better consumers, you obviously want the information you are providing them to be accurate.  One type of information is quality ratings of health care providers.  So how good are those ratings?  Researchers writing in the New England Journal of Medicine Catalyst attempt to answer that question.  (NEJM Article)   Six experienced methodologists were asked to assess the rating systems.  They looked at four rating systems for hospitals; CMS’ Hospital Compare, one done by US News and World Report, the Leapfrog rating system and Healthgrades.  Six key factors were used in the evaluation:  the potential to misclassify hospital performance; the importance or impact of the rating; scientific accuracy; iterative improvement; transparency and usability.  To cut to the chase, these ratings systems basically suck.  US News and World Report came in at the top, with a B grade, CMS got a C (no comment about a government product necessary), Leapfrog a C minus and HealthGrades a D plus, also known as a go to your room and don’t come out for dinner grade.  On the most important factor, correctly classifying hospital performance, US News & World Report was judged to be the best.  Among the factors touched upon by the authors that create issues with the ranking systems are the use of limited data and lack of robust data audits; inadequately validated measures of quality; lumping different hospital types together and no peer review of the methods used for compiling the ratings.  Some of these systems also may have had financial conflicts as they let hospitals pay them for the right to display their rating.  Their notes on the CMS hospital Star system are particularly useful since that system is widely used and visible, and has financial consequences for hospitals participating in Medicare.  They found the CMS system transparent and usable, and that its measures are created through a good process, but CMS lumps all hospitals together and there are issues both with the measures used and the weighting of those measures.

Looking at the ratings would be confusing to consumers; how do they decide which system to trust when they have different rankings.  So it is pretty clear to me, and was before this article, that you shouldn’t place a lot of trust in any of these rating systems; that they really aren’t going to tell you much about the quality of care you will receive.  And I don’t know what the point is anyway; we have allowed so much consolidation of health systems that there isn’t really much choice available to consumers.  These ratings systems also get little visibility compared to the immense marketing and advertising dollars spent by hospitals trying to convince consumers, usually successfully, of their quality.  That advertising, especially by non-profit systems, should be banned.  Those saved dollars might help reduce prices, which would be a real benefit to patients.