Improving the Reliability of Physician "Report Cards"
The reliability of individual primary care physician profiles was the focus of this study.
An evaluation of two possible ways to improve the reliability of individual primary care physician (PCP) performance profiles determined that neither combining measures into single or multi-disease composites, nor combining PCPs by practice groups, improved reliability for all patient conditions.
The authors conducted a cross-sectional analysis of electronic health record data for patients in an academic-affiliated health system with diabetes, congestive heart failure, ischemic vascular disease or who are eligible for preventive care services. They attempted to improve PCP profiles by combining condition-specific measures measures into single- and multi-disease composites and by profiling PCP practice groups.
Most PCPs could not be profiled with 80 percent reliability (the conventionally expected level of reliability). Combining measures for each condition into condition-specific composites improved reliability for diabetes and preventive care, but not for congestive heart failure or ischemic vascular disease. Combining PCPs by practice groups did not improve reliability for diabetes, congestive heart failure, or ischemic vascular disease.
Combining measures by disease, or into multi-disease composites, can improve reliability for some conditions, as a result of increased sample size. Evaluating PCPs by practice groups did not improve reliability because there was more variation between individual PCPs than between practice groups. The authors suggest that overall composites might be useful for comparison between providers or groups.