Evaluation of Voices for Healthy Kids Program

For too many children, the prospects for good health are limited by where they live or the unequal opportunities available to them.

A source of health inequity for many children is the level to which they are—or are not—affected by policies addressing childhood obesity. Many states and communities differ greatly when it comes to the policies they have enacted to prevent or reverse childhood obesity.

Voices for Healthy Kids®, a collaboration between the Robert Wood Johnson Foundation (RWJF) and the American Heart Association (AHA), aims to educate the public and policymakers about what is needed to prevent childhood obesity.

Its goal is to improve health equity and reduce childhood obesity by evaluating enacted and proposed policies; determine which ones work best; educate the public and policymakers of their success; and advocate their further adoption. The initiative awards grants to advocacy organizations already active in childhood obesity to encourage evidence-based regulatory and legislative activity.

Program evaluations are available for the first two years of the grant, and will be updated over time to reflect evaluations from the program’s third and future years.

Evaluation Overview

RWJF believes in the importance of capturing and sharing key data and lessons from its grantmaking—what worked, what didn’t, and why—and funds third-party evaluations to help assess impact and share lessons learned. The evaluations are intended to refine current grant programs, and inform future grantmaking. This evaluation was overseen by Laura Leviton, senior advisor for evaluation at RWJF, and led by George Grob, project director, of the Center for Public Program Evaluation, and Ruth Anne Gigliotti, the deputy project director of Synthesis Professional Services. Evaluation studies were conducted by evaluation teams from Johns Hopkins University, Harvard University, Barker Bi-Costal Health Consultants, Inc., Gretchen Swanson Center for Nutrition, and Portland State University. The Evaluation Committee was co-chaired by Monica Vinluan, senior program officer at RWJF, and Mark Schoeberl, executive vice president for advocacy and health quality at AHA.

Many of the evaluations are available to download on this webpage, and explore questions including:

• How much movement has there been on state-based policy as it relates to the program’s goals?
• What have we learned about the nature and effectiveness of the program’s technical assistance?
• How can advocacy most effectively be organized and managed to spur health policy change?
• How do media outlets cover childhood obesity and systemic causes and solutions?
• Which factors facilitated or hindered the progress of Voices-supported policy campaigns?

Broad “Policy Movement”

Trends in state legislation

• Nineteen bills were enacted in the baseline year (from November 2012 through October 2013).
• Thirty-four bills were enacted in the following year and 24 two years later.
• Sixty-five percent of bills related to a priority area in the baseline year.
• This increased to 75 percent during the following year and decreased back to 65 percent two years later.
• Bills with a health equity component increased from 34 percent in the baseline year to 42 percent in the follow-up periods.

Technical Assistance

This section includes four reports outlining technical assistance requests, processes, and user/client feedback; one case study highlighting how technical assistance helped support policy wins; and an evaluation of legal technical assistance focused on promoting effective advocacy for health policies while avoiding lobbying activities. As part of the feedback on technical assistance, a survey of AHA’s advocacy grantees identified message research and polling, media advocacy and communications, and legal assistance as particularly useful resources.

Reports and case studies for download:

• Technical Assistance: Client Engagement Survey of Advocacy Grantees
• Technical Assistance: User Feedback Interviews
• Technical Assistance: Concepts, Processes, and Management Systems
• Technical Assistance Requests for the Voices for Healthy Kids Initiative
• Technical Assistance Case Study: Vermont Sugar-Sweetened Beverage Excise Tax Campaign
• Evaluation of Legal Technical Assistance

Policy Advocacy for Social Transformation

Focusing on the central question of how can advocacy most effectively be organized and managed to spur health policy change, these reports are constructed as a resource for health policy researchers, community advocates, and others intending to be involved with health policy advocacy, and as a resource guide for those who lead or participate in policy advocacy campaigns.

Reports available for download:

• Policy Advocacy for Social Transformation—An Introduction and Practical Guide
• Advocacy Readiness: Measurement Tools and Resources for Advocacy Groups

Newspaper Coverage and Media Advocacy

Evaluators reviewed news articles published from September 1, 2012 through September 1, 2014 by 15 prominent U.S. newspapers to capture the tenor of the coverage of childhood obesity. Of the stories analyzed, 277 stories included information on the systemic causes or solutions to the problem, with the biggest stories covering the national rollout of federal school nutrition standards enacted under the Healthy, Hunger-Free Kids Act of 2010. First Lady Michelle Obama’s national Let’s Move! Campaign also drove coverage during this time.

Reports available for download:

• Speaking of A Culture of Health: Childhood Obesity Framing in the News
• Media Advocacy and Framing Principles—Practical Guides for Telling the Story of Child Healthy Weight and School Foods within a Culture of Health
  • Metaphor: Replacing a zero-sum war with a shared journey
  • Social Math: “Metaphor for numbers” to advance a Culture of Health
  • Authentic Voices: Engaging youth in the issues that affect them
  • Letters to the Editor: An important Media Advocacy tool to advance a Culture of Health

After Action Reviews

Evaluators explored which factors facilitated or hindered the progress of Voices-supported policy campaigns. Hindrances to these campaigns were most heavily concentrated in the political and fiscal climate, while campaign progress was expedited by experienced advocates; cohesive coalitions or partnerships; cultivating relationships with and support from policymakers/decision-makers; and marshaling grassroots participation.

Report available for download:

• Cross-Review of First Ten After Action Reports

Year Three and Beyond—Ongoing Evaluation Reports, Implementation and Impact

Ongoing evaluations will round out the assessment of the key structures of Voices for Healthy Kids and look to the future of the program and to other complementary policy initiatives.

Projects underway include:

• Further progress in enactment of state policies aimed at reducing childhood obesity; and
• Progress in implementation of state and local government childhood obesity policies.

Beyond year three, evaluations will continue to focus on implementation of enacted policies and the impact they have had to date. This will likely entail ongoing evaluation and research over the course of several years.

About Voices for Healthy Kids

Voices for Healthy Kids is a collaborative initiative of the Robert Wood Johnson Foundation and the American Heart Association (AHA) to reverse childhood obesity through public education and advocacy. It awards grants to advocacy organizations already active in childhood obesity to encourage evidence-based regulatory and legislative activity with the greatest potential for impact. It also promotes policies to improve health equity. Voices grantees are prohibited from using grant support from RWJF for lobbying. However, AHA does provide some grantees with lobbying funds.