Study Evaluates the Feasibility and Quality of Online Patient Surveys Rating Physician Performance

    • March 16, 2004

During 2002 and 2003, researchers at the non-profit advocacy organization FACCT—the Foundation for Accountability (no longer in existence)—assessed the feasibility of using data from on-line patient surveys to create a valid assessment of physicians' performance. The research team collected and analyzed data from two main sources:

  • Patient surveys.
  • Focus groups of patients and physicians.

The project was the third phase of a five-phase research project funded by the Robert Wood Johnson Foundation (RWJF) designed to test, refine and validate this Internet-based health information survey tool for consumers, called "CompareYourCare"™.

Key Findings

  • As reported to RWJF:

    • The survey yielded sufficient data to calculate a reliability of the findings for portions of the "CompareYourCare" survey dealing with three areas of physicians' care:
      • Service & Support—which includes measures for "quality of physician-patient interactions," "access and continuity of care" and "integration of care."
      • Appropriate Care, based on national treatment guidelines.
      • Education & Teamwork, which measures the support provided to the patient by the physician.
    • Survey respondents rated their physician's Education and Treatment much lower if they were being given adult general health care, than if their treatment was for chronic conditions.
    • Survey respondents rated physicians that provide gynecological care higher for three categories—Support and Service: Access and Continuity of Care, Integration of Care and Quality of Physician-Patient Interactions—than family practitioners and internists.
    • Patients in the focus groups stated a preference to use their doctor's score to work with him or her to improve the quality of care and communication.
    • Both patients and doctors, in separate focus groups, expressed concern about the scientific integrity of quality profiles on individual doctors, patient confidentiality, how the Internet will be used to collect and distribute health survey data, and how scores will be used to reward/punish providers and patients.
    • Both doctors and patients asserted that a person's physician is the best messenger to solicit survey participation.

Key Conclusions

  • In a report to RWJF, project researchers stated the following conclusions regarding the future use of a patient survey to develop quality of care profiles for individual doctors:

    • The federal Health Insurance Portability and Accountability Act of 1996 (HIPAA) may pose an insurmountable barrier to research such as that attempted in this project.
    • A significant number of adults (26.3%) living in a computer and Internet "dense" region of California report that they do not have a computer and/or Internet access.