October 1998

Grant Results

SUMMARY

There is growing concern in the United States that the way Americans die in hospitals is over-medicalized and contrary to patient preferences. Health care providers want to respond to these concerns, but lack the methodology to measure how well they are taking care of dying people.

George Washington University's Center to Improve Care of the Dying convened a panel of 31 experts to review current knowledge about measuring quality of care at the end of life and develop a consensus on a draft toolkit of measurement. The two-day "Conference on Measuring Care at the End of Life" was held in August 1996 in Woods Hole, Mass.

Key Results

  • The draft toolkit, based on the perspectives of dying patients and their families, contained an executive summary, annotated bibliographies of existing measurement instruments, recommendations of measurement instruments to be considered for use, and several new measurement tools.
  • The experts outlined an agenda for further research on patient and family perspectives, health care institution accountability, and cost-effectiveness of obtaining information on quality of care at the end of life.
  • The draft toolkit was made available to health care institutions and researchers through a Brown University Web site.
  • The principals published articles in the Journal of the American Geriatrics Society and Alzheimer Disease and Associated Disorders and in the Journal of Pain and Symptom Management and Hospice Journal.
  • IOM, used the draft toolkit's executive summary and new measures in its report Approaching Death: Improving Care at the End of Life.
  • The draft toolkit was used in the Robert Wood Johnson Foundation's (RWJF) national program Promoting Excellence in End-of-Life Care.
  • At the close of this project, RWJF-funded follow-up grants (see Grant Results on ID#s 032208 and 033491, 038149 and 041478) developed a validated version of the toolkit, an accompanying resource guide, and software for analysis and reporting data.

Funding
RWJF supported the cost of attendance for 15 participants ($43,770). Remaining costs were funded by the Nathan Cummings Foundation, and by the Institute of Medicine, (IOM) through a grant from The Commonwealth Fund.

 See Grant Detail & Contact Information
 Back to the Table of Contents


THE PROBLEM

The way in which Americans die has changed profoundly over the past century. In 1900, people usually died within days or weeks of becoming ill, at home, surrounded by family members. Now, the majority of people die in hospitals and long-term care institutions, often after a protracted illness lasting several months or several years.

Despite this prolongation of decline, little attention has been devoted to examining and improving the quality of life of the dying person. Existing quality-of-care instruments focus on patient function, but do not measure other domains such as emotional symptoms or family outcomes.

Interest in measuring and improving the quality of care for dying persons was generated by the national debate over physician-assisted suicide (and Oregon's debate over the possible legalization of physician-assisted suicide) and previous work funded by RWJF.

 Back to the Table of Contents


RWJF STRATEGY

The national initiative called Program on the Care of Critically Ill Hospitalized Adults, run by George Washington University's ICU Research Unit, was RWJF's first undertaking in this area. It conducted the Study to Understand Prognoses and Preferences for Outcomes and Risks of Treatments (SUPPORT) from 1989 to 1994, which studied decision-making and outcomes of illnesses for seriously ill adults and hospitalized patients age 80 and older.

SUPPORT identified a number of important opportunities to improve care of patients with a limited life expectancy and increased interest among hospitals and health care professionals in measuring and improving care of dying patients.

Since funding SUPPORT, the Foundation has continued funding in the area of improving care of dying patients through continued funding of SUPPORT, support of three additional national programs, Promoting Excellence in End-of-Life Care, Community-State Partnerships to Improve End-of-Life Care, and Targeted End-of-Life Projects Initiative, a communications campaign called Last Acts™, as well as a large number of additional grants.

 Back to the Table of Contents


THE PROJECT

This grant provided funds to George Washington University to convene experts in the measurement of quality of care at the end of life and to establish a consensus on the elements of a "toolkit" to help health care providers assess the quality of the care they provide to terminally ill patients.

The grant had four objectives:

  • Critically evaluate known measurements regarding the quality of care at the end of life using the framework proposed by the Medical Outcomes Trust's Scientific Advisory Committee of the Medical Outcomes Trust, (a U.S.-based, nonprofit repository of health care measurement tools), and create a comprehensive resource guide to current measures.
  • Reach consensus on a recommended set of measures and procedures — a user-friendly toolkit — that institutions can use now to measure quality of care at the end of life.
  • Disseminate available and recommended measures to health care institutions and researchers through direct mail, professional publications, professional organizations, and the World Wide Web.
  • Identify priorities for future research to improve measures that reflect the needs of the public, health care providers, and policymakers.

In order to accomplish objectives one, two, and four, the center held a conference on Measuring Care at the End of Life on August 27 and 28, 1996, in Woods Hole, Mass.

RWJF supported the attendance of 15 out of 31 multidisciplinary experts (participants are listed in the Appendix). The center received the remaining support needed from the Nathan Cummings Foundation, and from the Institute of Medicine, (IOM) through a grant from The Commonwealth Fund.

Before the conference, researchers at the Center to Improve Care of the Dying completed comprehensive literature reviews of available measurement instruments, examining the 11 domains proposed by the American Geriatrics Society:

  • Advance care planning.
  • Continuity of care.
  • Emotional symptoms.
  • Functional status.
  • Grief.
  • Family burden.
  • Pain and physical symptoms.
  • Quality of life.
  • Satisfaction.
  • Spirituality.
  • Survival time.
  • Aggressiveness of care.

As planned, they used criteria proposed by the Medical Outcomes Trust's Scientific Advisory Committee. These literature reviews were sent to the conference participants and their input was incorporated into a briefing book.

During the conference, the experts agreed that measures must be:

  • Clinically meaningful and manageable in their application.
  • Focused on the middle manager who wants to assess the institution's quality of care for dying patients, strengths, and opportunities for improvement.
  • Responsive to both the patient and family.

In addition they agreed:

  • Medical record review of people who have died must be considered as another important perspective.
  • Rapid cycles of improvement in measurement should occur, with the first toolkit quickly becoming outdated with research and further development of guidelines.

They also drafted a set of research priorities. This project did the groundwork for developing tools and measures of quality of End-of-Life care, and-in purposefully addressing patient and family perspectives-represented a first step in developing measures relevant to the needs and expectations of the dying and their families.

Communications

Objective number three was fulfilled through a variety of dissemination methods.

  • Journal articles have been published in the Journal of the American Geriatrics Society and Alzheimer Disease and Associated Disorders, and in the Journal of Pain and Symptom Management and Hospice Journal.
  • Presentations were made to the Agency for Health Care Policy Research and the Gerontological Society of America, among others. The Web site at Brown University, on which the briefing book was posted, has an important link on several clearinghouse sites devoted to hospice care and the study of death and dying.
  • Growth House, the Web site that provides a resource clearinghouse for the dying community, gave the site a two-star award and highlighted it as an important resource in a monthly newsletter.
  • More than 300 copies of the briefing book were mailed upon request. See the Bibliography for complete details.

 Back to the Table of Contents


RESULTS

  • An annotated Bibliography of existing instruments to assess the quality of care at the end of life was created. A briefing book was prepared containing an annotated Bibliography of existing instruments, with reviewers' recommendations on instruments to be considered in the toolkit, and a key article reporting on the psychometric properties of that instrument.
  • A draft set of instruments to assess the quality of care at the end of life was prepared. New measurement tools developed included a chart audit tool and patient and family interviews on quality of care for multiple locations of care (for patients dying in the hospital, hospice, or under the care of an integrated health care system). User-friendly instructions were provided with each instrument.
  • A draft toolkit to assess the quality of care at the end of life was developed. An executive summary was added to the annotated Bibliography and draft set of instruments to comprise the draft toolkit. The draft toolkit was placed on the World Wide Web site of the Center to Improve Care of the Dying at George Washington University.
  • Five research priorities and important design questions for future development of valid and reliable survey and chart review instruments were set. These included:
    1. What are the proper domains that define quality of care from the patient and family perspective? Are health care institutions accountable for the outcomes in a particular domain?
    2. What are the time frames in which quality of care should be assessed?
    3. Who is the respondent? On what is the respondent able to accurately report?
    4. What is the cost-effectiveness of various strategies to get information on the quality of care?
    5. How are the respondents' views influenced by the wording and location of the questions in the survey?
  • The draft toolkit was used in important health care initiatives. The IOM used the draft toolkit's executive summary and draft instruments in its report "Approaching Death: Improving Care at the End of Life," and the Institute of Health Care Improvement, a Boston-based, independent nonprofit organization working to improve health care systems in the United States, Canada, and Europe, used it as part of its initiative on quality of care at the end of life. The Institute of Health Care used the draft toolkit in its initiative on quality of care at the end of life. RWJF currently is using the draft toolkit in its national program Promoting Excellence in End-of-Life Care."

 Back to the Table of Contents


LESSONS LEARNED

  1. Including both academics and practice professionals in a project provides a necessary balance; academics ensure that methodology is sound, and practice professionals ensure that the project results are relevant and useful to the field.

 Back to the Table of Contents


AFTER THE GRANT

RWJF provided four grants totaling $1,096,220 to follow-up on the work conducted here. From 1997 to 2004, Joan M. Teno, M.D., M.S., led a team of researchers from Brown University and other institutions in creating a tool kit of instruments to measure the quality of care for dying patients and their families. See Grant Results on ID#s 032208, 033491, 038149 and 041478.

Two working conferences in 1997 and 2000 attended by experts in end-of-life care helped the researchers identify the essential areas for quality end-of-life care and map out a research agenda. Project staff developed two new instruments-a patient interview and a bereaved family member interview-designed to measure the care institutions give at the end of life.

Project staff tested and validated the family member interview instrument (but discontinued work on the patient interview) and created a Web interface where institutions can download this instrument, collect and submit data and receive reports for use in efforts to improve the quality of their end of life care.

 Back to the Table of Contents


GRANT DETAILS & CONTACT INFORMATION

Project

Developing a Toolkit to Help Hospitals Measure Quality of Care at the End of Life

Grantee

George Washington University (Washington,  DC)

  • Amount: $ 43,770
    Dates: August 1996 to June 1997
    ID#:  029719

Contact

Joan M. Teno, M.D., M.S.
(401) 863-1560
joan_teno@brown.edu

Web Site

http://www.chcr.brown.edu/pcoc/toolkit.htm

 Back to the Table of Contents


APPENDICES


Appendix 1

(Current as of the time of the grant; provided by the grantee organization; not verified by RWJF.)

Participants in Conference on Measuring Care at the End of Life

Carla Alexander, M.D.
Chase Brexton Health Services

Ira Byock, M.D.
University of Montana

Carolyn J. Cassin
Hospice of Michigan

Myra J. Christopher
Midwest Bioethics Center

Brian Claridge, Ph.D.
University of Massachusetts at Boston

Inge B. Corless, R.N., Ph.D.
Massachusetts General Hospital

J. Randall Curtis, M.D., M.P.H.
University of Washington

Molla S. Donaldson
Institute of Medicine

Robert Dunlop, M.D.
St. Christopher's Hospice

Susan Edgman-Levitan
The Picker Institute

Betty Ferrell, Ph.D.
City of Hope National Medical Center

Frank D. Ferris, M.D.
Mount Sinai Hospital

Marilyn Field, Ph.D.
Institute of Medicine

Jack Fowler, Ph.D.
University of Massachusetts at Boston

Steve Franey
Foundation for Accountability

Suzy Goldhirsch
Mount Sinai Medical Center

Audiey Kao, M.D.
American Medical Association

Barbara A. Kreling
George Washington University Medical Center

Joanne Lynn, M.D., M.A.
George Washington University Medical Center

Neil Mac Donald, C.M., M.D.
McGill University

Melanie Merriman, Ph.D.
VITAS Healthcare Corporation

Naomi Naierman, M.P.A.
American Hospice Foundation

David W. Rehm, A.C.S.W.
Hospice Care of Rhode Island

Bonnie Ryan, R.N.
Department of Veterans Affairs

True Ryndes
San Diego Hospice

Emily Schifrin, M.S.
National Committee for Quality Assurance

Barry Smith
VistaCare

Mildred Z. Solomon, Ed.D.
Center for Applied Ethics

Marguerite Stevens, Ph.D.
Dartmouth-Hitchcock Medical Center

Anita Stewart, Ph.D.
Stanford University

Joan Teno, M.D., M.S.
Brown University

 Back to the Table of Contents


BIBLIOGRAPHY

(Current as of date of this report; as provided by grantee organization; not verified by RWJF; items not available from RWJF.)

Articles

Byock IR, Teno JM, and Field MJ. Editorial, "Measuring Quality of Care at Life's End." Journal of Pain and Symptom Management, 17(2): 73–74, 1999.

Fowler F, Coppola KM, and Teno JM. "Methodological Challenges for Measuring Quality of Care at the End of Life." Journal of Pain and Symptom Management, 17(2): 114–119, 1999. Abstract available online.

Rudberg MA, Teno, JM, and Lynn J. "Developing and Implementing Measures of Quality of Care at the End of Life: A Call for Action." Journal of the American Geriatrics Society, 45(4): 528–530, 1997.

Stewart AL, Teno J, Patrick DL, and Lynn J. "The Concept of Quality of Life of Dying Persons in the Context of Health Care." Journal of Pain and Symptom Management, 17(2): 93–108, 1999. Abstract available online.

Teno JM. "Looking Beyond the 'Form' to Complex Interventions Needed to Improve End-of-Life Care." Journal of the American Geriatrics Society, 46(9): 1170–1171, 1998.

Teno JM. "Putting the Patient and Family Voice Back into Measuring Quality of Care for the Dying." Hospice Journal, 14(3–4): 167–176, 1999. Abstract available online.

Teno JM, Byock IR, and Field MJ. "Research Agenda for Developing Measures to Examine Quality of Care and Quality of Life of Patients Diagnosed with Life Limiting Illness." Journal of Pain and Symptom Management, 17(2): 75–82, 1999. Abstract available online.

Teno JM and Coppola KM. "For Every Numerator, You Need a Denominator: A Simple Statement But Key to Measuring the Quality of Care of Dying." Journal of Pain and Symptom Management, 17(2): 109–113, 1999. Abstract available online.

Teno JM and Landrum KA. "What is the Quality of Quality of Care Measurement for Dying Patients?" In progress.

Teno JM, Landrum K, and Lynn J. "Defining and Measuring Outcomes in End-Stage Dementia." Alzheimer Disease and Associated Disorders, 11(Suppl. 6): 25–29, 1997. Abstract available online.

Presentations and Testimony

Joan M. Teno. "Defining and Measuring Outcomes in Dementia — But What About the Family?" at the Alzheimer's Association Conference "Defining and Measuring Outcomes in Alzheimer's Research: Do We Agree?," Washington, D.C., September 1996.

Joan M. Teno. "From the Classroom to Clinical Practice," at the Spirituality and Medical Education Curricular Development Conference, Washington, D.C., April 25, 1997.

Joan M. Teno. "Quality of Care: Role for Ethics Committees?" at the Kaiser Permanente Southern California Ethics Committee Retreat, Los Angeles, Calif., April 30, 1997.

Joan M. Teno. "Measuring Quality of Care for Persons Diagnosed with Life Limiting Illness," at the Agency for Health Care Policy Research, Rockville, Md., June 9, 1997.

Joan M. Teno. "What Should the American Cancer Society Do in End of Life Care?" at the American Cancer Society, Atlanta, Ga., June 9, 1997.

Joan M. Teno. "For Every Numerator, You Need a Denominator: A Simple Statement but Key to Measuring the Quality of Care of the 'Dying,'" at the Conference on Excellent Care at the End of Life through Fastracking Audits, Standards, and Teamwork, Woods Hole, Mass., September 28–30, 1997.

Joan M. Teno. "Consumer Reports and Ratings of Medical Care: When Will we Get Satisfaction?" at the Annual Meeting of the Gerontological Society of America, Washington, D.C., November 1997.

World Wide Web Sites

www.gwu.edu/~cicd/toolkit/toolkit.htm provides an annotated Bibliography of the toolkit of instruments developed from this grant, including an executive summary, background information on each domain, and a draft set of instruments. George Washington University, Center to Improve Care of the Dying, 1996.

www.chcr.brown.edu/pcoc/toolkit.htm provides an annotated Bibliography of the toolkit of instruments developed from this grant, including an executive summary, background information on each domain, and a draft set of instruments. Brown University, Center for Gerontology and Health Care Research, October 1997.

 Back to the Table of Contents


Report prepared by: Lori De Milto
Reviewed by: Janet Heroux
Reviewed by: Molly McKaughan
Program Officer: Rosemary Gibson