Public Health Practice: Evaluating the Impact of Quality Improvement

Adopting QI methods to improve the delivery of services

Thirteen health departments implemented 32 quality improvement (QI) projects and worked with evaluators to assess the results, gather lessons, and build evidence on the use of QI in public health and as a tool to prepare for accreditation.

Dates of Program: July 2008 to March 2012

Description: At the time RWJF funded this program, there was little published evidence on the value and impact of quality improvement (QI) in public health, and only preliminary evidence about the factors that promote effective implementation and spread of QI efforts in governmental health departments.

The program’s goal was to build the evidence of what works and what does not in public health QI.

Most health departments focused their QI projects on specific public health needs, such as immunizations, food safety, or family planning. Two health departments worked on department-wide QI. Their evaluations focused on the process and outcomes of the implementation of the QI project.

Participants used standard quality improvement approaches, such as the plan-do-study-act model (PDSA), and evaluated 56 metrics, such as immunization rates, number of restaurants with critical food safety violations, improved clinic flow, and QI attitude and knowledge.

Challenges: Evaluators for each health department used different evaluation approaches. Those differences, and the inconsistent quality of the evaluations, led to major problems synthesizing and disseminating the results. The national program office was unable to meet its major goal of using the evaluations to develop a Web-based compendium of best practices.

Key Results

  • The 32 QI projects resulted in significant improvements, based on the 56 metrics evaluated, according to the national program director, William Riley, PhD.

  • Project staff reported that their QI projects were effective and that participating in Public Health Practice: Evaluating the Impact of Quality Improvement improved their health departments’ QI culture.

  • A review of the individual health department project evaluations by an evaluation consultant resulted in the following recommendations:

    • Evaluation methods can be used to improve QI project planning and to understand what worked well in a specific QI project.
    • Evaluation is necessary to measure agency advancement to a culture of QI. Literature from health care and public health suggest that developing a culture of QI is most likely to result in sustained performance and improved health outcomes.