Skip to main content
Robert Wood Johnson Foundation
  • Our Vision
    • Our Vision Overview
    • Why Health Equity
    • Focus Areas
    • Measuring RWJF Progress
  • Grants
    • Grants Overview
    • Active Funding Opportunities
    • Awarded Grants
    • Grantee Stories
    • Grant Process
    • Grantee Resources
  • Insights
    • Insights Overview
    • Blog
    • Our Research
    • Advocacy And Policy
  • About RWJF
    • About RWJF Overview
    • Our Guiding Principles
    • How We Work
    • Impact Investments
    • Staff And Trustees
    • Press Room
    • Careers
    • Contact Us
    • Accessibility Statement
Find A Grant
Global Search Dialog
    Robert Wood Johnson Foundation
    • Our Vision
      • Our Vision Overview
      • Why Health Equity
      • Focus Areas
      • Measuring RWJF Progress
    • Grants
      • Grants Overview
      • Active Funding Opportunities
      • Awarded Grants
      • Grantee Stories
      • Grant Process
      • Grantee Resources
    • Insights
      • Insights Overview
      • Blog
      • Our Research
      • Advocacy And Policy
    • About RWJF
      • About RWJF Overview
      • Our Guiding Principles
      • How We Work
      • Impact Investments
      • Staff And Trustees
      • Press Room
      • Careers
      • Contact Us
      • Accessibility Statement
    Find A Grant
    Global Search Dialog

      Aligning Forces for Quality: Program Evaluation 2006 - 2016

      Evaluations and Assessments Aug-21-2020 | 4-min read
      1. Insights
      2. Our Research
      3. Aligning Forces for Quality: Program Evaluation 2006 - 2016
      American Journal of Managed Care: 2016 Special Issue - open access

      About RWJF's Aligning Forces for Quality (AF4Q)

      Aligning Forces for Quality, a national “signature effort” of the Robert Wood Johnson Foundation (RWJF), was launched in 2006. The program was designed to help communities across the country align health care forces to improve the quality of health care of 37 million people, providing models for widespread national reform. Over the course of a decade, RWJF’s investment in AF4Q provided unprecedented funding and expert technical assistance to 16 multi-stakeholder health care coalitions (“alliances”) composed of three groups: 1) those who get care; 2) those who give care; and 3) those who pay for care—to work together to lift the overall health care of their communities.

      AF4Q Goals / Objectives

      In each AF4Q community, a regional alliance of doctors, patients, consumers, insurers, and employers worked collaboratively to transform their local health care system. Lessons from these transformations would then be used to develop national models for reform. Alliances were tasked with addressing five “forces” to enhance quality while reducing costs:

      1. Performance measurement and public reporting
      2. Quality improvement
      3. Engaging consumers in their health and health care
      4. Reducing health care disparities
      5. Reforming payment

       

      Evaluating AF4Q

      The AF4Q evaluation was led by RWJF’s Senior Adviser for Evaluation, Laura C. Leviton (2005–2008), and Senior Program Officers Claire Gibbons (2008-2014), and Carolyn Miller (2014–2017). Dennis Scanlon, PhD, Director of the Center for Health Care and Policy Research at the Pennsylvania State University oversaw the evaluation team comprised of researchers from the Pennsylvania State University, the University of Michigan, the University of Minnesota, Northwestern University, and George Washington University. Evaluating this 10-year effort, the evaluation team examined the AF4Q program design and implementation as the national health care environment was undergoing ‘tectonic change,’ including the passage of the Affordable Care Act (ACA) in 2010.

      Evaluation Methodology

      The AF4Q evaluation was a longitudinal mixed-methods program evaluation (i.e., multiphase triangulated evaluation) of 16 grantee “alliances” from across the country. RWJF continuously adjusted their approaches and program goals as a result of the evaluation team’s findings, monitoring efforts of the AF4Q national program office and RWJF, and changes in national health care policy. Results were shared in journal articles, reports, briefs, presentations, (many available on rwjf.org), and social media posts over the course of the evaluation in the hopes that sharing lessons learned early on would allow those working in other health and health care settings to begin applying them to improve services within their own communities.

      Key Evaluation Findings

      • Implementing and aligning activities across all five “forces” proved difficult for any one AF4Q community.
      • While collaboration among stakeholders to improve systems is conceptually appealing, there are many challenges to sustaining regional relationships, though AF4Q highlighted some promising models.
      • Over time, the AF4Q regions showed no major differences in improvement rates in quality and health measures examined, compared with control communities. However, all communities trended toward improvement.
      • Engaging consumers in reforming the system proved more difficult than expected, but the conversation has been elevated. The importance of transforming the system “by patients and for patients” is gaining traction.
      • Sustaining quality and price transparency at the community level is challenging. Important questions moving forward include: Who will fund and produce this information, and who receives the most value from it?
      • Disparities reduction activities proved challenging but happened with more intensity in communities that had long-established relationships with community groups representing diverse and underserved populations.

      Lessons Learned

      With the exception of a small proportion of outcomes, there were no major differences in the rate of longitudinal improvement in AF4Q communities, compared with control communities, on quantitative outcomes related to the triple aim as defined by the Institute for Healthcare Improvement:

      • Improving the patient experience of care (including quality and satisfaction);
      • Improving the health of populations; and
      • Reducing the per capita cost of health care.

      Although the majority of the measures improved in both AF4Q and non-AF4Q communities, there were some exceptions to this improving trend, most noticeably in the cost of care and population health. There was also considerable heterogeneity across communities in terms of programmatic areas and the scale and scope of interventions in these areas. Although a number of AF4Q alliances implemented robust interventions in specific areas—often advancing strategies useful for others in the field—no AF4Q alliance pursued and aligned all five AF4Q programmatic areas in a robust way. In addition, while all alliances were able to garner the participation of multiple stakeholders initially, sustaining this participation and securing new sources of funding after RWJF support ended proved challenging for many alliances.

      Conclusion

      Although the AF4Q program did not attain the ambitious community-level changes predicted by RWJF at the program’s outset, it did produce pockets of success on some dimensions for particular alliances. A number of factors explain the less-than-expected impact of the AF4Q initiative on community health and the observed variation in alliance sustainability and intervention strength. These include: differing acceptance of the AF4Q initiative’s theory of change; variation in the experience and capacity of the alliance communities selected for the program; differences in alliances’ local health care market context; and the changing programmatic requirements for alliances participating in the AF4Q initiative. The variation in AF4Q program outcomes offers important lessons for those engaged in regional health improvement work.

      Related Content

      Library
      A group of professionals discuss research findings during a meeting.

      Evaluations and Assessments

      Browse a listing of evaluations of Robert Wood Johnson Foundation programs that assess effectiveness and impact and capture key lessons learned from our grantmaking.

      Subscribe to receive Funding Alerts & more

      Explore the latest in reflection and research from subject matter experts at RWJF and our wide network of partners.

      Email address already subscribed. Please check your inbox to manage your subscriptions.

      Subscribed!

      Thank you. You are now subscribed.

      Tell us what type of content you want to receive.

      Be informed with our twice a month newsletter updating you with relevant news and research around a Culture of Health, as well as the latest funding opportunities.

      Get funded by RWJF: Receive notifications when new funding opportunities are released.

      Receive monthly updates on RWJF-sponsored research that informs many robust health policy debates on Capitol Hill, covering topics like health equity, improving access to quality healthcare, equitable housing, and more.

      Shop talk for researchers. This monthly newsletter covers research news and opportunities from the Robert Wood Johnson Foundation.

      Communications expedite action. Get periodic research and expert insights on the best ways to communicate so we can spread what works.

      Sometimes we have news, announcements or opportunities that don't quite fit the subscription parameters above. If you're interested, we'll send you this information under "There's more...". *If you've indicated you are an EU resident, we will only send these communications if you intentionally check this box.

      Which profession or pursuit best describes you?

      Area(s) Of Interest

      Unsubscribe

      Stop receiving all emails from RWJF

      This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
      • CONTACT RWJF

        50 College Road East
        Princeton, NJ 08540-6614

         

        US Toll Free: (877) 843-7953

        International: +1 (609) 627-6000

      • MANAGE YOUR GRANTS

        MyRWJF Login

        • Facebook
        • LinkedIn
        • YouTube
        • Instagram

      • Our Vision
        • Our Vision Overview
        • Why Health Equity
        • Focus Areas
        • Measuring RWJF Progress
      • Grants
        • Grants Overview
        • Active Funding Opportunities
        • Awarded Grants
        • Grantee Stories
        • Grant Process
        • Grantee Resources
      • Insights
        • Insights Overview
        • Blog
        • Our Research
        • Advocacy And Policy
      • About RWJF
        • About RWJF Overview
        • Our Guiding Principles
        • How We Work
        • Impact Investments
        • Staff And Trustees
        • Press Room
        • Careers
        • Contact Us
        • Accessibility Statement

      ©2001- 

      Robert Wood Johnson Foundation. All Rights Reserved. 

       

      • Manage Email
      • Privacy Statement
      • Terms and Conditions