Evaluating a Ten-Year Effort to Transform Health Care

Sep 21, 2016, 3:20 PM, Posted by

An evaluation of RWJF’s quality improvement initiative, Aligning Forces for Quality, uncovers barriers that health collectives face and how successful cultural shifts have helped transform health systems.

AF4Q report map

I believe a unique opportunity for a philanthropic organization is to explore the big ideas. We have a freedom that few others have to really experiment and innovate. Even to take big risks in our grant making sometimes. Often, the rewards are insights they offer a glimpse of how our nation can address some of the most pressing challenges facing our society.  Aligning Forces for Quality is an example of this philosophy in action.”  —Risa Lavizzo-Mourey, MD, MBA, president and CEO of the Robert Wood Johnson Foundation 

In 2006, the Robert Wood Johnson Foundation (RWJF) launched a bold, ten year experiment that became one of its largest philanthropic investments—the Aligning Forces for Quality (AF4Q) initiative which sought to lift the overall quality, equality, and value of health care in 16 communities across the country.

In each AF4Q community, a regional alliance of doctors, patients, consumers, insurers, and employers worked collaboratively to transform their local health care system. Lessons from these transformations were then used to develop national models for reform. Alliances were tasked with addressing five “forces” to enhance quality while reducing costs:

  • performance measurement and reporting
  • quality improvement
  • engaging consumers in their health and health care
  • reducing health care disparities
  • reforming payment

Continuous Learning

Throughout its history, RWJF has placed a premium on evaluation to understand our work and its impact. So in addition to providing grants and technical assistance to the 16 alliances, RWJF funded an independent, scientific evaluation of the AF4Q program.

Led by Dennis Scanlon, PhD, director of the Center for Health Care and Policy Research at the Pennsylvania State University, the evaluation team includes researchers from the Pennsylvania State University, the University of Michigan, the University of Minnesota, Northwestern University, and George Washington University who are assessing the impact of AF4Q and identifying key lessons.

AF4Q was designed so that we could learn from it continuously as it evolved.

We constantly adjusted our approaches and goals based on findings of the evaluation team, monitoring efforts of the AF4Q National Program Office and RWJF, and changes in national health care policy. We shared our results in journal articles, reports, briefs, presentations, and social media posts throughout the ten-year journey in the hopes of providing valuable insights to all those who are working to improve health care at the community level.

New Insights

In August 2016, the American Journal of Managed Care published an online supplement featuring research reports and commentaries that take stock of AF4Q now that the final phase of the evaluation is concluding.

A key finding within the supplement stems from the evaluation team’s efforts to determine if AF4Q improved population health and health care quality measures. The evaluators found no major improvements in these measures within AF4Q communities compared with control regions.

Despite this, AF4Q has yielded tremendously valuable insights regarding how to improve these measures in the future. Donald Berwick, MD, MPP, former administrator of the Centers for Medicare & Medicaid Services (CMS), underscored this point in his letter as guest editor of the supplement: “As a result of RWJF’s courageous investment, we know far more now than ever about the nature, scale, and variation of the barriers that face American collectives that want to produce health for their communities.”

AF4Q also produced numerous successes, many of which are described in the supplement. For instance, almost all the AF4Q alliances increased consumer access to data on ambulatory quality and patient experience in their communities. In addition, most of the alliance leaders reported that AF4Q sparked a major cultural shift within their organizations in which the consumer perspective became more important and better integrated into their work.

Looking to the Future

In the coming months, RWJF will reflect on all that we’ve learned from AF4Q and how it will shape our work going forward. To facilitate this process, we’ll be conducting a series of after-action reviews a method for extracting lessons from one project or event and applying them to others. We expect to glean rich insights that will inform our health care programs and our initiatives focused on improving health equity, creating healthy communities, and engaging the for-profit sector to promote health. As a learning organization, we're committed not only to acquiring knowledge, but to changing our behavior to reflect new insights.    

For example, we have already gained knowledge about how to improve cross-sectoral partnerships, and in particular how to make partnerships with employers work better. And we have discovered that stakeholders in AF4Q communities valued learning from each other more than from outside experts. As a result, in the future we will place an even greater emphasis on helping local stakeholders network with one another.

We look forward to sharing how we apply these and other lessons to our vision for building a Culture of Health that will enable all Americans to live healthier lives. 

Anne Weiss

Anne F. Weiss is a managing director at the Robert Wood Johnson Foundation, leading efforts to build a culture of health by transforming health and health care systems. Read her full bio.





Brian Quinn

Brian Quinn, PhD, is the assistant vice president of Research-Evaluation-Learning (REL) at the Robert Wood Johnson Foundation. Read his full bio.