This article examines the accuracy of tools to identify lower-cost physicians. Many proposed health reforms rely on the identification of physicians who provide lower-cost services for a given condition. These reforms include programs that direct patients to these lower-cost physicians or reward physicians believed to deliver lower-cost care. However, no rigorous evaluation has assessed whether the tools used to identify lower-cost physicians are consistently accurate.

The authors analyzed data from 2004 to 2005 from four health plans in Massachusetts. Their data set included 12,789 doctors and 1.1 million patients with claims. The researchers determined the cost per patient of specific disease treatment episodes. They then compared episode costs across physicians in a specialty to determine which physicians were associated with lower and higher than average costs. Finally, these results were compared with traditional cost-assessment tools to see whether traditional tools accurately captured cost habits of individual physicians.

Key Findings:

• More than one-fifth of physicians were incorrectly classified by cost-profiling tools. Among the physicians classified as lower-cost by profiling tools, 43 percent were not actually lower-cost.
• The reliability of cost-profiling tools varied widely by specialty: vascular surgery had the least reliable categorizations while gastroenterology and otolaryngology had the most reliable categorizations.
• A majority (59%) of physicians had cost-profiles with reliability under .70, an accepted benchmark for substandard reliability.

These findings suggest that there are significant problems with traditional tools to identify lower-cost doctors. The frequency of inaccurate categorization may present problems for reforms that aim to control health care costs by steering patients to physicians believed to provide lower-cost care.