Consumer Voices for Coverage

Advocacy Evaluation Toolkit

In 2008, through Consumer Voices for Coverage, the Robert Wood Johnson Foundation (RWJF) funded coalitions of consumer organizations in 12 states to advocate for public policies that would expand health insurance coverage. The Foundation made grants to a lead organization in each state and required each grantee to partner with other organizations that represented health care consumers to decide what types of public policies to support, build their capacity to conduct advocacy, and develop and implement advocacy strategies.

The Foundation funded Mathematica Policy Research to evaluate the Consumer Voices for Coverage program. This Advocacy Evaluation Toolkit is a product of the evaluation. It consists of the data collection tools Mathematica developed and used in the evaluation, which ended in 2011.

Mathematica developed the toolkit in response to Consumer Voices for Coverage grantees’ interest in strengthening their ability to evaluate their own organizations and advocacy efforts in the future. The toolkit is mainly intended to help the grantees and other advocacy organizations collect and analyze data using the instruments and methods Mathematica used in its evaluation. Therefore, it is designed for people who might not be familiar with evaluation procedures and methods. Given the challenges of evaluating advocacy (Coffman 2009; Raynor et al. 2009), evaluators and organizations that fund advocacy might find the instruments contained in the toolkit helpful.

The Advocacy Evaluation Toolkit contains the instruments Mathematica used to collect data for evaluating the Consumer Voices for Coverage program. It explains how the instruments were developed, what each was designed to measure, and how Mathematica used them for the evaluation. Although the instruments in the toolkit were designed to collect data for the grant program and reflect its structure and goals, they can be adapted for other situations and uses, ranging from an organization’s informal self-assessment to shape its activities to a comprehensive evaluation. The toolkit suggests some of these adaptations. For people who might not be familiar with evaluation methods, the Toolkit also describes some basic points on how to plan and conduct an evaluation.