Evaluation of Fighting Back

Evaluation conducted March 1994 through October 2002

About the Program

The Fighting Back® program started by the Robert Wood Johnson Foundation (RWJF) in 1988, was a $87.9 million national program to assist communities of 100,000 to 250,000 people to implement a variety of anti-drug strategies. It ran until the fall of 2003.

Fighting Back addressed drug problems through a community-wide approach, involving business, health care, the public school system, local government and its agencies, the police, community groups, local media, and the clergy.

About the Evaluation

There were two evaluation groups during the life of Fighting Back. The first, Pacific Institute for Research and Evaluation (PIRE), served from 1990 to 1994. In March 1994, Leonard Saxe, Ph.D., who was then at the City University of New York (CUNY) Graduate Center, assembled a new, interdisciplinary group of evaluators. In November 1995, the CUNY evaluators released an interim report of preliminary findings. In 1995, after Saxe moved to the Brandeis University Florence Heller Graduate School for Advanced Studies, Brandeis became the lead grantee for the evaluation. As of October 31, 2002, evaluation grants totaling $14.2 million have been awarded.

Because no single data source is a perfect measure of program effects, the evaluation gathered data from varied sources to maximize the validity and utility of its findings. The evaluation sought to determine if Fighting Back communities have been able to achieve measurable reduction in the overall use of or demand for alcohol and/or illegal drugs when compared to non-funded comparison communities.

Summary of Methods
The evaluation collected a broad range of qualitative and quantitative data from various sources about the nature and impact of the interventions in each of the 12 Fighting Back communities. Evaluators then compared these data in a quasi-experimental design with similar data from 29 comparison communities not funded under the Fighting Back program. Some of the comparison communities, however, were funded by the federal Community Partnership program, which was loosely modeled on Fighting Back. The data were analyzed preliminarily in 1995; and again in 1997, 1999 and 2002.

Data sources included:

  • Community indicators of drug and alcohol problems and responses at the community level, such as substance abuse-related deaths, traffic accidents, and criminal justice system statistics.
  • General population surveys of more than 40,000 people in the Fighting Back and comparison communities, (500 to 1,000 each from 12 Fighting Back communities and from the 29 comparison sites), designed, tested, and conducted by Schulman, Ronca, and Bucuvalas of New York City from 1994 to 1999.
  • Data derived from school-based surveys of adolescent substance abuse.
  • Ethnographic studies (including site visits, interviews, and observation of project activities, ethnographic detail, and social networks) designed to illuminate the social context surrounding Fighting Back initiatives and the process by which they operate.

Knowledge and Impact:

The program contributed to a movement for federal funding of a program based on similar aims of Fighting Back. More than 330 communities responded to RWJF's first call for proposals in 1989. The large number of communities competing for Fighting Back funding persuaded the Office of National Drug Control Policy (ONDCP) to employ the Fighting Back model in an expanded, federally sponsored, national initiative called the Community Partnership Demonstration Grant (CPDG) program. In 1990 and 1991, the federal Center for Substance Abuse Prevention (CSAP) funded 251 partnerships from a total budget of $375 million.

The evaluation by Brandeis University concluded that community coalitions were not sufficient to change substance abuse problems. This is in spite of Fighting Back's sustained, 10-year community-based coalition approach that had ample technical assistance and direction, top-notch people, and sites that were pre-selected, but did not produce robust results in terms of decreasing substance abuse. One conclusion is that community coalitions alone are not a sufficient solution to the substance abuse problem.

At the same time, Fighting Back did demonstrate some accomplishments in effective coalition-building and a community's greater awareness of treatment, shown across sites. This result led Fighting Back's national program director, David Rosenbloom of Join Together, to conclude that helping communities develop more substance abuse treatment slots should be the main emphasis of RWJF's substance abuse efforts going forward. As a result, RWJF has funded a new Join Together initiative to help communities in this regard.

The evaluation uncovered another part of the Fighting Back story: confirming other information, the substance abuse problems in these inner-city sites turned out not to be dramatically worse than they were in the suburbs. Illegal drug supply is high in these kinds of neighborhoods, but much of the demand is coming from outside. This is an important insight of the program and has been applied in other more recent RWJF initiatives, such as the Urban Health Initiative, where the program attempts to work with a broader range of communities.

RWJF program staff pointed to positive effects that appeared in the evaluation in individual neighborhoods. For example, one neighborhood saw declines in the measures of alcohol dependency risk and alcohol dependence; another site saw declines in the measures for 30-day alcohol binge drinking, 12-month illicit drug use, and 12-month marijuana use; and a third site saw declines in the measures of 12-month marijuana use, knowledge of a treatment facility, and 12-month attendance at alcohol and other drug meetings. Yet, evaluator Leonard Saxe says that the data do not support even this modestly positive interpretation. He has a number of reasons for this conclusion:

  • The number of significant outcomes in specific sites is only slightly higher than the number expected by chance; when an evaluation looks at 100 different outcomes, the expectation is that by chance 5 or 10 will be positive, and the sites have not shown more positives than that.
  • If the few outliers among the people surveyed on the outcome measures are statistically adjusted, the positive effects disappear.
  • In a few of the sites, where there are significant effects on some measures in relation to the comparison site, the positive impact is associated more with unusual changes in the comparison site than with changes in the Fighting Back site.
  • If one looks at the activities each site concentrated on and then looks at the effects, they often do not match up; for example, a site that concentrated on the reduction of alcohol dependency has experienced reductions in the use of cocaine.
  • A number of the positive site-specific findings are concentrated in sites that were closed before the end of the full demonstration.

For all these reasons, the evaluator rejects the idea that the program has had even a modest impact in some sites.

In concurrence, the program evaluation's technical advisory committee stated that, even if there were some positive results, RWJF cannot widely disseminate them unless they can be replicated. The bottom line, according to Jim Knickman, then vice president of Research and Evaluation at RWJF, is that the results indicate that the Fighting Back approach does not offer a solution to the substance abuse problem. It may offer hope that something can be done through coalitions, but what the social science field and the substance abuse field want to know is: Is this a national strategy to tackle substance abuse? The indicators of substance abuse across the sites did not decrease sufficiently for RWJF to be able to say that Fighting Back's approach should be heralded as the solution.

Legislators should not think that spending money on community coalitions alone will be enough to solve such a pressing problem. Yet, coalitions may be a necessary piece of any effective strategy, and through the Fighting Back program, many lessons were learned about how to initiate and sustain community coalitions.