The Program on Chronic Mental Illness

Individuals with chronic mental illness face enormous challenges getting the care they need. Not only do they have to cope with their illness, but they and their families must also try to arrange services from two complex and often unreceptive systems of care: the medical system and the social service system. Following the movement of patients from state mental hospitals to community settings, the problems faced by people with chronic mental illness became more visible to the American public.

In 1987 and 1988, The Robert Wood Johnson Foundation launched three national programs to improve services for people with chronic mental illness. Two of them--the Mental Health Services Program for Youth, which was examined by Leonard Saxe and Theodore Cross in last year's Anthology, and the Program on Chronic Mental Illness, which is discussed in this chapter--attempted to better coordinate mental health care services. The third program, the Mental Health Services Development Program, did not focus on improving systems of care but, rather, on improving the quality of clinical and social services.

Howard Goldman, a professor of psychiatry at the University of Maryland, directed a large evaluation of the Program on Chronic Mental Illness. In this chapter, he reviews the successes and the failures of the program. First, however, he places the program within the context of the broad campaigns to reform the treatment of mentally ill individuals and, more narrowly, within the context of the Foundation's efforts in this field.

Often, evaluators are more negative about a program than the Foundation's staff or the national program office, who naturally feel pride and ownership of a program. In this case, the opposite occurred: Goldman and his team interpreted the results more positively than many of those involved in designing and running the program. He argues that those who were disappointed with the findings had unreasonable expectations or focused on the wrong outcomes. Perhaps more important, he highlights the limitations of an approach to treating chronic mental illness that concentrates on improving systems of delivering services and ignores the quality of those services. The lessons from the Foundation-funded mental health programs about the importance of improving both clinical services and systems of care were applied in later mental health research and services programs financed by the federal government.

Chronic mental illness has been a social welfare policy issue in the United States for almost two centuries. Characterized by a series of reform movements, mental health policy has been focused on a variety of organizational solutions to the problems of individuals with severe mental illness.

Policy analysts and historians have described four cycles of reform. The first cycle, during the early nineteenth century, championed "moral treatment" in asylums; the second, almost a century later, advanced scientific "mental hygiene," to be practiced in psychopathic hospitals and clinics. The first two reform cycles expanded a variety of public and private institutions for treating people with mental illness. Before World War II, most services were delivered in large state and county asylums, and in a smaller number of private hospitals and clinics.

After World War II, a third cycle of reform, financed largely by the federal government and state mental health authorities, shifted the focus to community-based care in private offices and community mental health centers. Outpatient services expanded greatly during this period, employing new treatments, including antipsychotic and antidepressant medications. These changes ushered in the era of deinstitutionalization, in which the resident populations of state and county mental hospitals were reduced dramatically--from a high of about 560,000 in 1955 to well below 100,000 by the 1990s. Inpatient and long-term residential services in other settings, especially general hospitals and nursing homes, substituted for much of this dramatic decline. Mental health care policies and resources associated with deinstitutionalization, however, were not accompanied by commensurate changes in social welfare policies and resources--in income support and housing subsidies, for example--to accommodate the shift to community care. Individuals with mental illness, particularly those with severe and chronic mental disorders such as schizophrenia and manic-depressive illness, were not as well integrated into their communities as the reformers had expected.

Each of these reform movements proposed a new way of treating mental illness in a new setting. Each theorized that early intervention in acute cases would prevent the problems posed by chronic mental illness. For 150 years, pursuing these strategies resulted in the continuing neglect of people with severe mental illness.

Then, in the 1970s, a fourth cycle of reform proposed a program of community support to address the problems of chronic mental illness. Instead of placing continued hope in early intervention, the community support movement recharacterized the problem of chronic mental illness in social welfare as well as psychiatric terms. This approach broadened the scope of policy to include issues of employment, income support, transportation, and housing. In doing so, the movement revealed the fragmentation of the system of services needed by those with chronic mental illness. Policy makers asked, Who is responsible for this population, and what organizational strategies might succeed in meeting their complex needs?¹

The Foundation entered the world of mental health services in the mid-1980s--well into its fourth cycle of reform, the cycle that introduced the community support system and an expanded focus on the social welfare needs of those with chronic mental illness. In addition to mental health services, community support systems included rehabilitation and general medical services, housing support, case management, transportation, and social welfare assistance in obtaining income support, entitlements, and family support. By the mid-1980s, the ideology and the framework for a decentralized community support system had gained widespread acceptance, but implementation fell short of its goals, particularly in large American cities.

Mental illness was prevalent among homeless persons served by sites in The Robert Wood Johnson Foundation Health Care for the Homeless program. Homelessness among mentally ill persons was seen as evidence of a failure in both the organization and financing of mental health services and in providing access to mainstream social welfare resources. In urban districts, the service system was fragmented, with no central point of accountability. People with chronic mental illness lacked access to social welfare resources, such as Social Security disability payments and rental subsidies from the Department of Housing and Urban Development.

After a period of analysis from 1984 to 1986, the Foundation concluded that the problem of mental health services was a systems problem, requiring intervention in the organization and financing of services.²It developed a series of three initiatives: the Mental Health Services Development Program, the Mental Health Services Program for Youth, and the Program on Chronic Mental Illness, the biggest of the three. All began in the late 1980s and continued into the 1990s.

Although the Foundation's activity in mental health slowed in the mid-1990s, the impact of these three initiatives was felt throughout the mental health services field, stimulating new research and new ways of looking at service systems. The initiatives also influenced federal and state mental health activities--especially a subsequent wave of demonstration research. Table 6.1 describes the key elements of these programs.

The Mental Health Services Program for Youth (MHSPY) and the Program on Chronic Mental Illness (PCMI) were designed to demonstrate the benefits of centralizing responsibility and authority for the care and treatment of children with severe emotional disorders and adults with severe mental illness. The complex problems of these two populations demanded a wide array of services from both the mental health system and the broader health and human services system. These systems were themselves fragmented, and, although they shared many clients, rarely collaborated effectively with each other. During the fourth cycle of reform, state and federal government innovators developed the conceptual framework of the community support system for delivering services to individuals with complex problems. Of course, various community support systems had been proposed and introduced in scattered communities in the United States and elsewhere. Although a small body of literature evaluating them had emerged in the early 1980s,³there was no established body of experience to draw upon to guide their widespread adoption.

MHSPY focused on integrating the broader human service systems in urban areas to support the development of systems of care for children with severe emotional disorders. Children and their parents were placed at the center of a system of services involving health care and mental health care, the schools, child welfare, and, when necessary, juvenile justice. A better-integrated system of services was expected to envelop the child and promote the continuity of care and improved outcomes. The evaluation of MHSPY concentrated on how strategies for systems integration were put into place: it did not evaluate individual child and family outcomes. This approach was consistent with the objectives of the demonstration, which emphasized systems change. It was assumed that positive outcomes would naturally result from integrated services and improved systems of care and "wrap-around" services.⁴

In similar fashion, the Program on Chronic Mental Illness focused on systems integration through the creation of Local Mental Health Authorities--governmental and quasi-governmental entities designed to centralize administrative, fiscal, and clinical responsibility for adults with chronic mental illness. It was believed that these would reduce fragmentation, introduce a community support system, improve the continuity of care, and promote improved clinical and social outcomes for individuals and their families.⁵

In contrast, the Mental Health Services Development Program provided grant resources to develop specialized case management, assertive community treatment (ACT) teams, and vocational support programs. ACT, a multidisciplinary team approach to providing intensive mental health services to severely disabled individuals living in community settings, was the prototype. Like other innovative community mental health services, ACT had been developed and evaluated in a few settings but not widely adopted. By sponsoring innovative programs at the level of direct service delivery, the Mental Health Services Development Program largely ignored trendy systems interventions to focus on improving clinical and social care services for individual clients with severe mental illness.⁶

In retrospect the Mental Health Services Development Program may have been the most forward-thinking of the three initiatives, although at the time it was viewed as the most prosaic and traditional. The Robert Wood Johnson Foundation devoted more resources to the systems demonstrations for adults and children, but the evaluations of the larger initiatives, particularly the PCMI, demonstrated the limitations of systems change alone and underscored the importance of improving client-level service interventions, such as those sponsored by the Mental Health Services Development Program.

Late in 1986, The Robert Wood Johnson Foundation provided $29 million in grants and loans to nine cities in the United States: Austin, Baltimore, Charlotte, Cincinnati, Columbus, Denver, Honolulu, Philadelphia, and Toledo. Additionally, the Department of Housing and Urban Development provided each city with 125 Section 8 certificates to subsidize rents for individuals with severe and persistent mental illness so that they could afford safe housing. A national program office, run by Miles Shore and Martin Cohen at the Harvard Medical School, provided the grantees with technical assistance in setting up a Local Mental Health Authority and developing housing in their cities.⁷

Often, these Local Mental Health Authorities were part of city or county government--or were private organizations with a board of directors appointed by a government agency. They planned mental health care delivery at each site, allocated resources, and provided clinical oversight. Some also provided mental health services directly. Each also created a housing development agency to help create housing opportunities for clients using Section 8 rent certificates from HUD. The Local Mental Health Authority assumed responsibility for the care of adults with mental illness, particularly for individuals with chronic mental illness. It was accountable for integrating the system of care to promote continuity of care and meet the complex needs of the target population.⁸

The Program on Chronic Mental Illness was a demonstration of systems change, and was not intended (or funded) to provide specific clinical and social services. It was expected, however, that state-of-the-art mental health services would be available at each of the sites. The logic of the program held that poor outcomes for those with severe mental illness were due to a lack of coordinated services. Systems change, in particular a better-integrated system of care with a Local Mental Health Authority at the center, would lead to more continuity of care, to greater availability of care (such as ACT), and ultimately to better outcomes for people with severe mental illness and their families. This was the systems integration hypothesis that the Program on Chronic Mental Illness was designed to demonstrate.

Initially, The Robert Wood Johnson Foundation was not interested in evaluating the impact of the Program on Chronic Mental Illness on individuals, preferring to focus on the systems-level objectives, as it did with the Mental Health Services Program for Youth. However, the National Institute of Mental Health (NIMH) was eager to support a more elaborate evaluation.

NIMH saw the Program on Chronic Mental Illness as an opportunity to test a number of strategies for research on mental health services, especially in the area of assessing individual outcomes. In particular, it wanted to evaluate the impact of systems change on individual mental health, including symptoms, social and occupational functioning, and quality of life. NIMH allocated $2 million to match the $2 million offered by the Foundation for the evaluation.

The Request for Proposals, jointly prepared by the Foundation's and NIMH's staffs, reflected the intent to evaluate the Program on Chronic Mental Illness from systems-level implementation through its impact on individuals with chronic mental illness and their families. Ten organizations were invited to submit proposals. The team of investigators selected to evaluate the program were from the University of Maryland, Johns Hopkins University, and the University of North Carolina--with additional consultants. It was a multidisciplinary team with expertise in the sociology of organizations, housing and urban studies, economics, and clinical and social outcomes research. Members of the evaluation team were awarded additional grants--one to study interorganizational networks, for example, and another to assess family burden--and collaborated with other investigators, who were separately funded to study results at other sites. This expanded evaluation gave the research team the ability to investigate the program in great detail and from multiple perspectives .⁹

The results of the evaluation, which extended for a year or so past the end of the program in 1992 have been published elsewhere. ¹⁰ However, several points can be made here about them and their impact on the mental health services field.

The evaluation concluded that Local Mental Health Authorities were feasible and desirable structures for centralizing administrative, fiscal, and clinical responsibility in large cities. Such authorities, however, were slow to be created and required technical assistance, even more than financial resources, to establish and strengthen them. They took many forms, adapting the basic concept to local conditions with good effect; no standard form could be recommended.¹¹

Individuals interviewed at each of the demonstration sites praised the Local Mental Health Authority. Measures of interorganizational network relationships (derived from detailed surveys of administrators at dozens of organizations at each site) demonstrated that the systems in the PCMI cities were less fragmented and more centralized, and reported more contact with one another as the demonstration progressed--and more so when compared to a nondemonstration site with a model mental health system but no Local Mental Health Authority.¹² Furthermore, demonstration sites with functioning Local Mental Health Authorities showed higher levels of continuity of care for people discharged from twenty-four-hour acute care settings.¹³

However, the program's impact on individuals and their families was mixed. The evaluation contained a quasi-experimental comparison of two cohorts of individuals with severe mental illness--one recruited near the beginning of the demonstration and followed for a year, the other recruited near the end. Outcomes were derived from well-validated interviews of the participants, with questions based mostly on symptoms, behavior, and experiences. The expectation was that the later cohort, exposed to the effects of improved systems, would have better outcomes than the earlier cohort, and that the improvement would reflect the positive impact of improved integration of services and continuity of care.

Consistent with the basic evaluation hypothesis, the cohorts observed late in the demonstration were more likely to have a case manager and to retain the same case manager throughout the follow-up period. The results on social and clinical outcomes did not meet expectations, however. There were no differences in clinical, social, and quality of life outcomes for cohorts of patients compared early and late in the demonstration, followed for a one-year period after discharge from twenty-four-hour care. Although most clients in both of the cohorts at each of the sites showed substantial improvement during the observation period, there were no statistically significant differences between the two groups.¹⁴

More consistent with expectations, family members of these clients found that their burden was reduced when the clients had case managers.¹⁵ Furthermore, a separate study demonstrated the benefits of receiving services directly. In Baltimore and Cincinnati, cohorts of clients who received HUD certificates to subsidize their rent and who received case management services showed measurable improvements in some areas, using almost identical measures to those used with the other cohorts.¹⁶

The findings on individual outcomes led to considerable discussion and a search for explanations, particularly given the positive findings on systems change. There was resistance to viewing the results as a refutation of the systems integration hypothesis, the basic premise of the program. Were the findings evidence that improvements in systems integration could not be expected to result in improved outcomes for those with chronic mental illness? The evaluators were reluctant to discount the positive findings about continuity of care, case management, and reduced family burden, and the positive outcomes for recipients of rent certificates.

The evaluators suggested that limitations in the design of the evaluation, such as the selection of a small sample derived only from hospitalized patients with varying exposure to the benefits of the program, might offer some explanation. Another possible reason for the mixed results could lie in the type of services received by the clients. A detailed retrospective survey of services at each of the sites indicated that the vast majority of the clients received the standard form of case management available at the sites, but that this case management was not the state-of-the-art care recommended for most individuals with chronic mental illness who had been discharged from twenty-four-hour care. In particular, assertive community treatment (ACT) was practically unavailable to the subjects in the client evaluation study.¹⁷

Many individuals, including staff members at The Robert Wood Johnson Foundation, emphasized the negative findings about the impact on clinical and social outcomes. The positive findings about the systems-level effects, including the feasibility of Local Mental Health Authorities and successes in developing independent housing, were largely ignored in favor of seeing the evaluation of the program only in terms of the failure to find social and clinical improvements for individuals with chronic mental illness. Even this was somewhat surprising in view of other positive findings about the program's impact on continuity of care, family burden, and individual outcomes for recipients of HUD rent certificates. Before the demonstration, HUD policy makers and program staff expressed doubt that individuals with chronic mental illness could use the Section 8 rent certificates effectively--let alone have improved outcomes when using them.

There are several possible explanations for such a narrow interpretation of the findings. A clue came early in the demonstration, when Foundation staff members pushed the evaluators to publish some findings from the continuing evaluation. The response was a special section of three papers in Hospital & Community Psychiatry describing the demonstration, its evaluation, and some preliminary findings, including the slowness of implementation in a few of the larger cities.¹⁸ An editorial that accompanied the articles, written by one of the reviewers of the manuscripts, concluded that the program represented nothing new and that its initial lack of success (based on delays in implementation) indicated a flawed design.¹⁹ It seemed that some people within the mental health field were willing to make decisions about the program even without data about its impact.

Other observers believed that the success of the demonstration could be assessed only in terms of the impact on people with chronic mental illness. They took the position that achievements at the systems level were unimportant if they were not accompanied by an impact on the clients.

The failure to find client-level impacts in the cohort comparison was uniform and unexpected. It attracted great attention, obscuring many of the individual-level positive findings. To a certain extent, the evaluators themselves were to blame for the misplaced emphasis. The finding that improvement in the organization of care made no difference in clients' mental health required an explanation--which the evaluators did not provide initially.

After eliminating methodological explanations for no-difference findings, the evaluators sought substantive answers. The repeated site visits had offered a possible explanation, as had site visits by members of the National Advisory Committee (NAC), after the program had been under way for two years. NAC members, especially the psychiatrists, returned from the sites concerned that the Local Mental Health Authorities lacked clinical plans. Although the Local Mental Health Authorities had developed administrative and financial plans, they had not given much thought to clinical services for clients. Instead, they relied primarily on ordinary case management approaches, which are typical of public sector mental health services. Initially, the Foundation staff, the Program on Chronic Mental Illness staff, and the evaluation staff considered the NAC observations to be a red herring, as the aim of the program was systems reform, not simply clinical services reform. As the program progressed, however, it became clear that the sites lacked clinical direction and had not adopted state-of-the-art services, such as ACT.

The evaluators concluded that systems change and the creation of a Local Mental Health Authority were necessary but not by themselves sufficient to produce individual benefits. This conclusion was reinforced by several additional observations: Individuals in the program who received specific enhancements in service, such as the Section 8 rent certificates and the attendant support services, did experience improved outcomes. In addition, at one of the sites, a well-defined decline in the availability of clinical services during a fiscal crisis was associated with a marked drop in individual outcomes.²⁰ Although the systems change strategy of the program had produced accountable Local Mental Health Authorities that centralized responsibility for the chronically mentally ill, had integrated services, and had developed independent housing, the sites did not experience dramatic improvements in client outcomes without special attention to the quality of mental health services.

Similar findings were produced by another important evaluation in the children's mental health field. Leonard Bickman's evaluation of a demonstration at Fort Bragg, North Carolina, found that systems change without enhancement of clinical services did not improve the mental health of patients.²¹

The content and quality of clinical and social services had been a neglected element in the Program on Chronic Mental Illness and other demonstrations of system change conducted during the same period. As a result of the failure of systems change to improve individuals' mental health, many in the field of mental health service development began to focus on quality-of-care research, clinical guidelines, and evidence-based practice. It also brought renewed efforts in research demonstrations to link changes in systems to changes at the level of individual clients.

Despite a narrow interpretation of the findings of its evaluation, the Program on Chronic Mental Illness demonstrated the feasibility and the utility of Local Mental Health Authorities in promoting the integration of service systems. The program also demonstrated that integration could have a positive impact on continuity of care, family burden, and housing status for individuals with severe mental illness. In addition, the Mental Health Services Program for Youth illustrated several models of integrating services for children with severe emotional disorders. Together with the Mental Health Services Development Program, the Program on Chronic Mental Illness underscored the importance of attending to the need for high-quality services, even in well-integrated service systems.

When the evaluators of the program concluded that integration of the service system was necessary but not sufficient to improve individual outcomes, they set in motion a series of activities in the development of mental health services--activities that influenced the design of subsequent government projects. They also influenced a new generation of mental health service activities funded by The Robert Wood Johnson Foundation.

Along with other trends in health services--managed care and a growing concern about outcomes, for instance--the findings from the Program on Chronic Mental Illness and other demonstration programs intensified interest in measuring the quality of care. This interest included attention to using services whose effectiveness has been demonstrated, the development of evidence-based treatment recommendations, and attention to the quality of clinical and social services in programs devoted to systems reform. The evaluators of the Program on Chronic Mental Illness themselves became involved in two federal projects extending their work into the new areas: the Access to Community Care and Effective Services and Supports project (ACCESS), and the Schizophrenia Patient Outcomes Research Team (PORT).

During the waning days of the Program on Chronic Mental Illness in 1992, a federal task force on homelessness and mental illness released a report; among the recommendations was a call for a service systems demonstration project to further test the systems integration hypothesis.²² The demonstration was to determine whether a small grant of resources to promote the integration of systems would result in better coordination among mental health providers and other human services providers and would promote residential stability and improved quality of life for individuals who were homeless and had a mental illness. ACCESS was designed with the Program on Chronic Mental Illness in mind.²³

Some members of the task force and the staff at the Center for Mental Health Services, the federal agency given responsibility for the new demonstration, were aware of the more nuanced view of the findings of the Program on Chronic Mental Illness. They encouraged ACCESS to provide substantial resources to the demonstration sites to develop clinical and social services, in addition to grants and technical assistance for promoting systems integration. Service grants of between $600,000 and $800,000 a year were made to each of the sites to develop Assertive Community Treatment outreach teams--the service viewed as missing in the Program on Chronic Mental Illness and presumed to be responsible for the no-difference findings. In this fashion, ACCESS became a demonstration of the integration of systems combined with improvement of services.

It was no accident that the Program on Chronic Mental Illness influenced the federal ACCESS program. In spite of a sense of competition between the federal mental health agencies and The Robert Wood Johnson Foundation, Foundation staff members had coordinated their efforts in the program with those of the government. Furthermore, the Foundation coordinated the funding and oversight of the evaluation with the National Institute of Mental Health and other federal agencies. Federal and state governments knew about the program, and were aware of its findings. Additionally, participants in the program and its evaluation served as consultants to the government on the design of ACCESS. Many of the evaluators of the program ultimately worked on its evaluation.

As ACCESS rolled out in 1993, the findings of the Program on Chronic Mental Illness were held out as lessons--particularly the need to develop state-of-the-art services at each site. This was critical to the design of ACCESS and its evaluation that compared the nine sites in each state receiving both additional resources for services integration and enhanced clinical services in the form of intensive case management--that is, ACT services--and the nine sites that received only resources to enhance clinical services. This strategy assesses whether the quality of services alone, or in concert with systems integration, leads to improvement in clinical and social outcomes.

Preliminary results indicate that, twelve months after first receiving services, previously homeless service recipients in sites with better integration of services are more likely to be independently housed than those in sites with less well-integrated services.²⁴ Studies of the impact of changes in integration over time, however, are not complete.

When the Agency for Health Care Policy and Research and the National Institute of Mental Health announced their intent to fund a Schizophrenia Patient Outcomes Research Team (PORT), a group of the evaluators for the Program on Chronic Mental Illness seized the opportunity and, with some new colleagues, successfully bid on the contract. This gave them the chance to apply the lessons learned from the evaluation of the Program on Chronic Mental Illness to treatment recommendations. Now in its sixth year, the team has reviewed the literature on the effectiveness of treatment, conducted field studies, and published treatment recommendations on schizophrenia. It has almost completed an assessment of the impact of efforts to disseminate the recommendations.²⁵

The PORT treatment recommendations coincided with a broader initiative within the mental health field to develop treatment guidelines and medication algorithms for use by psychiatrists and other practitioners. Several efforts at disseminating them have been initiated, among them the Texas Medication Algorithm Project (TMAP), that is supported by The Robert Wood Johnson Foundation and other funders.

ACCESS and PORT, as well as the original Program on Chronic Mental Illness, have underscored the importance of technical assistance in facilitating the implementation of a program; technical assistance in the design of Local Mental Health Authorities and on housing development were essential in the program. This lesson was relearned in ACCESS, in which the adoption of systems change was hampered by a lack of understanding of the central concept of service systems integration. After almost two years of a stuttering start, special resources were allocated to ACCESS project implementation. As a result, each of the ACCESS sites developed strategic plans for services integration and increased its level of integration activities. New demonstrations and efforts at disseminating treatment guidelines have made similar discoveries.

The emphasis on the content and quality of services has extended from ACCESS and the Schizophrenia PORT to a number of federal mental health demonstration projects. Programs focusing on employment services, supportive housing, treatment of simultaneously occurring mental illness and substance abuse, and diversion of individuals with mental illness from jail into treatment all underscore the importance of quality of care and the development of services. This emphasis is traceable to the lessons learned from the Mental Health Services Program for Youth, the Mental Health Services Development Program, and the Program on Chronic Mental Illness. Models of systems integration and service innovation, developed in these Foundation-supported programs, have been replicated in newer demonstration programs and have had a far-reaching impact.

1. G. Grob. The Mad Among Us. New York: Free Press, 1994; H. Goldman and J. Morrissey. "The Alchemy of Mental Health Policy: Homelessness and the Fourth Cycle of Reform," American Journal of Public Health, 1985, 75, 727-731. (return to article)

2. L. Aiken, S. Somers, and M. Shore. "Private Foundations in Health Affairs: A Case Study of a National Initiative for the Chronically Mentally Ill." American Psychologist, 1986, 41, 1,290-1,295.(return to article)

3. R. C. Tessler and H. H. Goldman. The Chronically Mentally Ill: Assessing Community Support Programs. Cambridge, Mass.: Ballinger, 1982.(return to article)

4. M. J. England and R. F. Cole. "Broad Continuum of Care Needed to Serve Mentally Ill Children," Psychiatric Times, 1990, 7(11); R. F. Cole, "Financing Policy and Administrative Mechanisms for Child and Adolescent Mental Health Services." The Child, Youth, and Family Service Quarterly, 1990, 13(2); R. F. Cole and S. Poe. Partnerships for Care. Washington, D.C.: Washington Business Group on Health, 1993; Final Progress Report, "Evaluation of the Mental Health Services Program for Youth Phase III," RWJF Grant #13613 Grant Period-February 1, 1991-June 30, 1995. RWJF: Princeton, N.J. (File).(return to article)

5. M. Shore and M. Cohen. "The Robert Wood Johnson Program on Chronic Mental Illness: An Overview." Hospital and Community Psychiatry, 1990, 41, 1,212-1,216; M. Shore and M. Cohen. "Introduction." Milbank Quarterly, 1994, 72, 31-35. (return to article)

6. L. I. Stein and M. A. Test. Alternatives to Mental Hospital Treatment. New York: Plenum, 1975; L. I. Stein, "The Robert Wood Johnson Foundation Mental Health Services Development Program," New Directions for Mental Health Services 45, 1990, 75-89; Final Report: "Mental Health Services Development Program January 1, 1987-August 31, 1991." Grant I.D. #16485. RWJF: Princeton, N.J. (File); L. I. Stein. "Innovative Community Mental Health Programs." New Directions for Mental Health Services 56, 1992, entire issue. (return to article)

7. H. Goldman, J. Morrissey, and M. S. Ridgely. "Form and Function of Mental Health Authorities at RWJ Foundation Program Sites: Preliminary Observations." Hospital and Community Psychiatry, 1990, 41, 1,222-1,230.(return to article)

9. H. Goldman, A. Lehman, J. Morrissey, S. Newman, R. Frank, and D. Steinwachs. "Design for the National Evaluation of the Robert Wood Johnson Foundation Program on Chronic Mental Illness." Hospital and Community Psychiatry, 1990, 41, 1,217-1,221. (return to article)

10. H. Goldman, J. Morrissey, and M. S. Ridgely. "Form and Function of Mental Health Authorities at RWJ Foundation Program Sites: Preliminary Observations." Hospital and Community Psychiatry, 1990, 41, 1,222-1,230; H. Goldman, J. P. Morrissey, M. S. Ridgely, and others. "Lessons from the Program on Chronic Mental Illness." Health Affairs, 1992, 11, 51-68; H. Goldman, J. Morrissey, and M. S. Ridgely. "Evaluating the Program on Chronic Mental Illness" (RWJ PCMI). Milbank Quarterly, 1994, 72(1), 37-48; J. Morrissey, M. Calloway, T. Bartko, M. S. Ridgely, H. Goldman, and R. Paulson. "Mental Health Authorities and Service System Change from the RWJ PCMI." Milbank Quarterly, 1994, 72(1), 49-80; A. Lehman, L. Postrado, D. Roth, S. McNary, and H. Goldman. "An Evaluation of Continuity of Care, Case Management, and Client Outcomes in The RWJ PCMI." Milbank Quarterly, 1994, 72(1), 105-122; M. S. Ridgely, J. Morrissey, R. Paulson, H. Goldman, and M. Calloway. "Case Management and Client Outcomes in the Robert Wood Johnson Foundation Program on Chronic Mental Illness." Psychiatric Services, 1996, 4(7), 737-743; J. A. Talbott. "Evaluating the Johnson Foundation Program on Chronic Mental Illness: An Interview with Howard Goldman." Psychiatric Services, 1995, 46(5), 501-503. (return to article)

11. H. Goldman, J. Morrissey, and M. S. Ridgely. "Form and Function of Mental Health Authorities at RWJ Foundation Program Sites: Preliminary Observations." Hospital and Community Psychiatry, 1990, 41, 1,222-1,230; H. H. Goldman, J. P. Morrissey, M. S. Ridgely and others, "Lessons from the Program on Chronic Mental Illness." Health Affairs, 1992, 11, 51-68; H. Goldman, J. Morrissey, and M. S. Ridgely. "Evaluating the Program on Chronic Mental Illness (RWJ PCMI)." Milbank Quarterly, 1994, 72(1), 37-48. (return to article)

12. J. Morrissey, M. Calloway, T. Bartko, M. S. Ridgely, H. Goldman, and R. Paulson. "Mental Health Authorities and Service System Change from the RWJ PCMI." Milbank Quarterly, 1994, 72(1), 49-80. (return to article)

13. A. Lehman, L. Postrado, D. Roth, S. McNary, and H. Goldman. "An Evaluation of Continuity of Care, Case Management, and Client Outcomes in the RWJ PCMI." Milbank Quarterly, 1994, 72(1), 105-122. (return to article)

15. R. Tessler and G. Gamache. "Continuity of Care, Residence, and Family Burden in Ohio." Milbank Quarterly, 1994, 72, 149-170.(return to article)

16. S. Newman, J. Rechovsky, K. Kaneda, and A. Hendrick. "The Effects of Independent Living on Persons with Chronic Mental Illness: An Assessment of the Section 8 Certificate Program." Milbank Quarterly, 1994, 72, 171-198. (return to article)

17. M. S. Ridgely, J. Morrissey, R. Paulson, H. Goldman, and M. Calloway. "Case Management and Client Outcomes in The Robert Wood Johnson Foundation Program on Chronic Mental Illness." Psychiatric Services, 1996, 4(7), 737-743. (return to article)

18. M. Shore and M. Cohen. "The Robert Wood Johnson Program on Chronic Mental Illness: An Overview." Hospital and Community Psychiatry 41, 1990, 1,212-1,216; M. Shore and M. Cohen. "Introduction." Milbank Quarterly, 1994, 72, 31-35; H. Goldman, J. Morrissey, and M. S. Ridgeley. "Form and Function of Mental Health Authorities at RWJ Foundation Program Sites: Preliminary Observations." Hospital and Community Psychiatry, 1990, 41, 1,222-1,230; H. Goldman, A. Lehman, J. Morrissey, S. Newman, R. Frank, and D. Steinwachs. "Design for the National Evaluation of The Robert Wood Johnson Foundation Program on Chronic Mental Illness." Hospital and Community Psychiatry, 1990, 41, 1,217-1,221. (return to article)

19. J. Rosenberger. "Central Mental Health Authorities: Politically Flawed?" Hospital and Community Psychiatry, 1990, 41, 1,171.(return to article)

20. D. Shern, N. Wilson, A. Coen, D. Patrick, M. Foster, D. Bartsch, and J. Demmler. "Client Outcomes II: Longitudinal Data from the Colorado Treatment Outcome Study." Milbank Quarterly, 1994, 72, 123-148. (return to article)

21. L. Bickman. "A Continuum of Care: More Is Not Always Better." American Psychologist, 1996, 51, 689-701. (return to article)

22. Federal Task Force on Homelessness and Mental Illness. Outcasts on Main Street (ADM 92-1904). Washington, D.C.: Interagency Council on the Homeless, 1992.(return to article)

23. F. Randolph, M. Blasinsky, W. Leginski, L. Parker, and H. Goldman. "Creating Integrated Service Systems for Homeless Persons With Mental Illness: The ACCESS Program." Psychiatric Services, 1997, 48, 369-373. (return to article)

24. R. Rosenheck, J. Morrissey, J. Lam, M. Calloway, M. Johnsen, H. Goldman, F. Randolph, M. Blasinsky, A. Fontana, R. Calsyn, and G. Teague. "Service System Integration, Access to Services, and Housing Outcomes in a Program for Homeless Persons with S evere Mental Illness." American Journal of Public Health, 1998, 88(11), 1,610-1,615. (return to article)

25. See Schizophrenia Bulletin, 1995, 21(4), for a series of articles on the issue; and A. Lehman, D. Steinwachs, and others. "Translating Research Into Practice: The Schizophrenia Patient Outcomes Research Team (PORT) Treatment Recommendations." Schizophrenia Bulletin, 1998, 24(1), 11-19. (return to article)