Physician Profiling Reliability

Rand researchers have published a study in the New England Journal of Medicine relating to the validity of methods used to evaluate the practice patterns of physicians, or more properly, the costs related to those practice patterns.  (NEJM Article) The authors examined claims data from four Massachusetts health plans and used commercially available software to build episodes of care and attribute those episodes to physicians.  They then used statistical methods to identify how reliable the categorization of physicians as high or low cost was likely to be and how often misclassification would occur.   They found that most physicians would have reliability scores below .7 and about 22% of doctors would be misclassified.

Because physicians control so much health spending, understanding their practice patterns,  in regard to services they personally deliver as well as those they order, such as prescriptions for drugs, devices and equipment, referrals to other health professionals, ordering of diagnostic tests and use of inpatient facilities, is a critical element of getting control of that spending.  A large body of prior research suggests that variations in practice patterns are not well correlated with better health outcomes for patients and may be influenced by doctors’ economic needs.  The current study is purely statistical, it suggests the mathematical likelihood that the ranking of physicians as low or high cost is accurate.  A better method would be to conduct a direct review of actual treatment patterns ordered by specific physicians on similar patients and compare that to the results of profiling.  That is obviously expensive and time-consuming, but it might validate use of profiling, instead of calling it into question, as a statistical approach will almost inevitably do.

The underlying issue is the difficulty of tracking through claims data what services a particular physician ordered and the problems in attributing all of a patients’ care to any specific physician.  This task is rendered more difficult when it is widely acknowledged that for many of the system’s highest cost, most complex patients, multiple physicians and facilities are often used in an uncoordinated manner.  It should also be noted that this difficulty in patient attribution to a physician mirrors the problems that will exist with episode or bundled payments–how do you decide who is in charge of the patient’s care?