Aligning Forces for Quality: Program Evaluation 2006 - 2016
Skip to main content
Robert Wood Johnson Foundation
  • Our Vision
    • Our Vision Overview
    • Why Health Equity
    • Focus Areas
    • Measuring RWJF Progress
  • Grants
    • Grants Overview
    • Active Funding Opportunities
    • Awarded Grants
    • Grantee Stories
    • Grant Process
    • Grantee Resources
  • Insights
    • Insights Overview
    • Blog
    • Our Research
    • Advocacy And Policy
  • About RWJF
    • About RWJF Overview
    • Our Guiding Principles
    • How We Work
    • Impact Investments
    • Staff And Trustees
    • Newsroom
    • Careers
    • Contact Us
Find A Grant

    Aligning Forces for Quality: Program Evaluation 2006 - 2016

    Evaluations and Assessments Aug-21-2020 | 4-min read
    1. Insights
    2. Our Research
    3. Aligning Forces for Quality: Program Evaluation 2006 - 2016
    American Journal of Managed Care: 2016 Special Issue - open access

    About RWJF's Aligning Forces for Quality (AF4Q)

    Aligning Forces for Quality, a national “signature effort” of the Robert Wood Johnson Foundation (RWJF), was launched in 2006. The program was designed to help communities across the country align health care forces to improve the quality of health care of 37 million people, providing models for widespread national reform. Over the course of a decade, RWJF’s investment in AF4Q provided unprecedented funding and expert technical assistance to 16 multi-stakeholder health care coalitions (“alliances”) composed of three groups: 1) those who get care; 2) those who give care; and 3) those who pay for care—to work together to lift the overall health care of their communities.

    AF4Q Goals / Objectives

    In each AF4Q community, a regional alliance of doctors, patients, consumers, insurers, and employers worked collaboratively to transform their local health care system. Lessons from these transformations would then be used to develop national models for reform. Alliances were tasked with addressing five “forces” to enhance quality while reducing costs:

    1. Performance measurement and public reporting
    2. Quality improvement
    3. Engaging consumers in their health and health care
    4. Reducing health care disparities
    5. Reforming payment

     

    Evaluating AF4Q

    The AF4Q evaluation was led by RWJF’s Senior Adviser for Evaluation, Laura C. Leviton (2005–2008), and Senior Program Officers Claire Gibbons (2008-2014), and Carolyn Miller (2014–2017). Dennis Scanlon, PhD, Director of the Center for Health Care and Policy Research at the Pennsylvania State University oversaw the evaluation team comprised of researchers from the Pennsylvania State University, the University of Michigan, the University of Minnesota, Northwestern University, and George Washington University. Evaluating this 10-year effort, the evaluation team examined the AF4Q program design and implementation as the national health care environment was undergoing ‘tectonic change,’ including the passage of the Affordable Care Act (ACA) in 2010.

    Evaluation Methodology

    The AF4Q evaluation was a longitudinal mixed-methods program evaluation (i.e., multiphase triangulated evaluation) of 16 grantee “alliances” from across the country. RWJF continuously adjusted their approaches and program goals as a result of the evaluation team’s findings, monitoring efforts of the AF4Q national program office and RWJF, and changes in national health care policy. Results were shared in journal articles, reports, briefs, presentations, (many available on rwjf.org), and social media posts over the course of the evaluation in the hopes that sharing lessons learned early on would allow those working in other health and health care settings to begin applying them to improve services within their own communities.

    Key Evaluation Findings

    • Implementing and aligning activities across all five “forces” proved difficult for any one AF4Q community.
    • While collaboration among stakeholders to improve systems is conceptually appealing, there are many challenges to sustaining regional relationships, though AF4Q highlighted some promising models.
    • Over time, the AF4Q regions showed no major differences in improvement rates in quality and health measures examined, compared with control communities. However, all communities trended toward improvement.
    • Engaging consumers in reforming the system proved more difficult than expected, but the conversation has been elevated. The importance of transforming the system “by patients and for patients” is gaining traction.
    • Sustaining quality and price transparency at the community level is challenging. Important questions moving forward include: Who will fund and produce this information, and who receives the most value from it?
    • Disparities reduction activities proved challenging but happened with more intensity in communities that had long-established relationships with community groups representing diverse and underserved populations.

    Lessons Learned

    With the exception of a small proportion of outcomes, there were no major differences in the rate of longitudinal improvement in AF4Q communities, compared with control communities, on quantitative outcomes related to the triple aim as defined by the Institute for Healthcare Improvement:

    • Improving the patient experience of care (including quality and satisfaction);
    • Improving the health of populations; and
    • Reducing the per capita cost of health care.

    Although the majority of the measures improved in both AF4Q and non-AF4Q communities, there were some exceptions to this improving trend, most noticeably in the cost of care and population health. There was also considerable heterogeneity across communities in terms of programmatic areas and the scale and scope of interventions in these areas. Although a number of AF4Q alliances implemented robust interventions in specific areas—often advancing strategies useful for others in the field—no AF4Q alliance pursued and aligned all five AF4Q programmatic areas in a robust way. In addition, while all alliances were able to garner the participation of multiple stakeholders initially, sustaining this participation and securing new sources of funding after RWJF support ended proved challenging for many alliances.

    Conclusion

    Although the AF4Q program did not attain the ambitious community-level changes predicted by RWJF at the program’s outset, it did produce pockets of success on some dimensions for particular alliances. A number of factors explain the less-than-expected impact of the AF4Q initiative on community health and the observed variation in alliance sustainability and intervention strength. These include: differing acceptance of the AF4Q initiative’s theory of change; variation in the experience and capacity of the alliance communities selected for the program; differences in alliances’ local health care market context; and the changing programmatic requirements for alliances participating in the AF4Q initiative. The variation in AF4Q program outcomes offers important lessons for those engaged in regional health improvement work.

    +

    SHARE

    Subscribe to receive Funding Alerts & more

    Explore the latest in reflection and research from subject matter experts at RWJF and our wide network of partners.

    Email address already subscribed. Please check your inbox to manage your subscriptions.

    Subscribed!

    Thank you. You are now subscribed.

    Tell us what type of content you want to receive.

    Be informed with our weekly newsletter updating you with relevant news and research around a Culture of Health, as well as the latest funding opportunities.

    Get funded by RWJF: Receive notifications when new funding opportunities are released.

    Receive monthly updates on RWJF-sponsored research that informs many robust health policy debates on Capitol Hill, covering topics like health equity, improving access to quality healthcare, equitable housing, and more.

    Shop talk for researchers. This monthly newsletter covers research news and opportunities from the Robert Wood Johnson Foundation.

    Sometimes we have news, announcements or opportunities that don't quite fit the subscription parameters above. If you're interested, we'll send you this information under "There's more...". *If you've indicated you are an EU resident, we will only send these communications if you intentionally check this box.

    Yes! I want to tell you more about myself and receive personalized communications in the future.

    Area(s) Of Interest

    Which profession or pursuit best describes you?

    Unsubscribe

    Stop receiving all emails from RWJF

    This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
    • CONTACT RWJF

      50 College Road East

      Princeton, NJ 08540-6614

       

      US Toll Free: (877) 843-7953

      International: +1 (609) 627-6000

    • MANAGE YOUR GRANTS

      MyRWJF Login

      • Twitter link
      • Facebook link
      • LinkedIn link
      • YouTube link

    • Our Vision
      • Our Vision Overview
      • Why Health Equity
      • Focus Areas
      • Measuring RWJF Progress
    • Grants
      • Grants Overview
      • Active Funding Opportunities
      • Awarded Grants
      • Grantee Stories
      • Grant Process
      • Grantee Resources
    • Insights
      • Insights Overview
      • Blog
      • Our Research
      • Advocacy And Policy
    • About RWJF
      • About RWJF Overview
      • Our Guiding Principles
      • How We Work
      • Impact Investments
      • Staff And Trustees
      • Newsroom
      • Careers
      • Contact Us

    ©2001- 

    Robert Wood Johnson Foundation. All Rights Reserved. 

     

    • Manage Email
    • Privacy Statement
    • Terms and Conditions