Foundations, as Michael Porter and Mark Kramer observed in an influential article published in 1999, are often ambivalent about evaluation. “The overall failure to evaluate the results of foundation grants is the most telling danger sign of all. Almost no money is set aside for program evaluation…[yet] without evaluation, a foundation will never know whether or not it has been successful.” Evaluation has become more important in the field of philanthropy since then, as foundation boards and staff members increasingly demand evidence of the impact of their grantmaking and as policy-makers and the public insist on more accountability.
The Robert Wood Johnson Foundation has placed a high priority on program evaluation since its inception as a national philanthropy in 1972. It has developed a four-tiered system of evaluation that ranges from the evaluation of individual grants and clusters of grants to the qualitative assessments found in The Robert Wood Johnson Foundation Anthology series.
In this chapter, two architects of the Foundation's evaluation system describe that system and examine its strengths and weaknesses. As the Foundation's vice-president for research and evaluation between 1992 and 2006, James R. Knickman was the person primarily responsible for the Foundation's approach to evaluation. Kelly A. Hunt, who was at the Foundation between 2000 and 2006, was a research officer and in charge of the Foundation's “Scorecard.” Knickman is currently the president and chief executive officer of the New York State Health Foundation, and Hunt is a senior program director there.