View all items in this collection

Program Results Report

An RWJF Collection

View all items in this collection

Determining Which Real-World Interventions Are Ready for More Rigorous Evaluation

Dates of Project: November 2006 through April 2010 and May 2009 through March 2014

Description: Early Assessment of Programs and Policies on Childhood Obesity was a project to identify, screen, and assess community programs and policies aimed at improving the diet and physical activity of children.

“We were seeing innovations popping up right and left like mushrooms after a spring rainstorm. But we lacked an understanding of what was feasible, what could be duplicated.”—Laura C. Leviton, PhD, Senior Advisor for Evaluation, RWJF

Researchers at the Centers for Disease Control and Prevention’s (CDC) Division of Nutrition, Physical Activity, and Obesity led the project along with a contract, ICF International, in consultation with a panel of experts from the federal government and other sectors. The initiative was the first full-blown test of the systematic screening and assessment (SSA) method, a process to identify and screen a high volume of real-world policy and environmental interventions that are potentially effective in preventing childhood obesity.

“Systematic screening and assessment wasn’t a methodology before this project. We said we should claim it, so other people could follow it.”—Laura Kettel Khan, PhD, Centers for Disease Control and Prevention

Key Results

  • The project team reported its results in a supplement published by New Directions in Evaluation:

    • Over a two-year period, the team identified 458 innovative programs in five priority areas, conducted evaluability assessments of 48 of those programs, and designated 20 as “highly promising.”
    • Project staff produced a synthesis report on each content area, documenting cross-site themes, challenges, and lessons emerging from the SSA process.

    The supplement received the American Evaluation Association’s Outstanding Publication Award in 2011. Project researchers cited these conclusions in the supplement and other journal articles:

  • The supplement received the American Evaluation Association’s Outstanding Publication Award in 2011. Project researchers cited these conclusions in the supplement and other journal articles:

    • SSA is a tool for discovering innovation. Many new childhood obesity interventions in real-world settings “might never have come to evaluators’ attention if the SSA method were not pursued as a first step in their evaluation.”
    • SSA is a cost-effective approach to ensuring more productive and useful evaluations. By rapidly identifying and assessing a high volume of interventions, SSA distinguishes promising programs from those that are not evaluation-ready.

    One of the programs identified by the SSA method was New York City’s regulations for licensed day-care centers, which designate the amount of time that children are to spend in physical activity, provide nutrition standards for foods and beverages on the premises, and place limits on screen time. CDC and ICF International worked with the New York City Department of Health and Mental Hygiene on the project, which ended in May 2014.

Citations:

Pitt Barnes S, Robin L, Dawkins N, Leviton L, Kettel Khan L. Early Assessment of Programs and Policies to Prevent Childhood Obesity Evaluability Assessment Synthesis Report: Comprehensive School Physical Activity Programs. Atlanta: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, 2009. Available online.

Pitt Barnes S, Robin L, Dawkins N, Leviton L, Kettel Khan L. Early Assessment of Programs and Policies to Prevent Childhood Obesity Evaluability Assessment Synthesis Report: Local Wellness Policy. Atlanta: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, 2009. Available online.

Wethington H, Hall MA, Dawkins N, Leviton L, Kettel Khan L. Early Assessment of Programs and Policies to Prevent Childhood Obesity Evaluability Assessment Synthesis Report: Childcare Initiatives in Afterschool and Daycare Settings. Atlanta: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, 2009. Available online.

Recommended

Share this

Shedding light on which interventions to combat childhood obesity are ready for rigorous evaluation.