July 2009

Grant Results

SUMMARY

During 2002 and 2003, researchers at the National Quality Forum were asked to facilitate the identification of topics for performance measure development useful for the consumer in selecting mammography centers. The project was designed as the preliminary phase of a study that would test the assumption that consumers will act on quality information to select better care, thus stimulating providers to compete by improving quality. However, in part because of results from this phase of the project, the larger study was never carried out.

The National Quality Forum is a private, nonprofit, open membership corporation established in 1999 for the primary purpose of developing standardized measures and reporting mechanisms to facilitate and support health care quality improvement.

Key Results

  • The project team identified six measures for further development that define quality mammography centers, focusing on the following topics:
    • Certification status.
    • Average turnaround time for reporting results.
    • Patient experience with care based on a standardized satisfaction survey.
    • Percentage of false-positive rates.
    • Percentage of false-negative rates.
    • Percentage of screening mammograms performed compared to all radiological procedures performed or as a percentage of all mammograms performed (screening and diagnostic) if at a comprehensive breast clinic.

Key Conclusions

  • There is a need for more heightened awareness among consumers and their primary care providers on the variation in quality of prevention/screening procedures.
  • In order to study how performance measurement data might alter the marketplace, it will be important to include a more fully diverse cross section of women, especially those with disabilities, as part of the discussion on consumer use of performance measures.
  • The potential measures identified in this project may require a level of data retrieval and coordination that is not consistently available across local, state or regional areas.

Funding
The Robert Wood Johnson Foundation (RWJF) supported this project with a solicited grant of $187,613.

 See Grant Detail & Contact Information
 Back to the Table of Contents


THE PROBLEM

Most existing market reforms to encourage health care quality rest on the assumption that consumers will act on quality information to select better care, stimulating providers to compete by improving quality. However, there is little direct evidence that consumers or providers will act this way.

Consumer decision-making relating to the reported quality and safety performance of mammography centers offers an opportunity to test this assumption because:

  • Mammography is a well-known screening test recommended for women between the ages of 40 and 75—women who tend to be actively involved in their health care.
  • It is a high-volume service (with some 9,000 certified mammography facilities in the United States performing more than 31 million mammography procedures per year, according to the U.S. Food and Drug Administration), making it relatively easy to distinguish differences in quality.
  • Although the implementation and enforcement of standards established by the 1992 Mammography Quality Standards Act have greatly improved and raised the bar for the quality and safety of mammography centers, there is still variation in the quality of mammography center practices.

 Back to the Table of Contents


RWJF STRATEGY

As noted in RWJF's 2002 Annual Report (the year in which RWJF issued the grant described in this report), "[r]esearch … illustrates that the quality of health services is often substandard, even though medical practitioners are learning how to implement cost-effective, high-quality care for many of the more problematic conditions." In 2002, RWJF issued grants to seven health care systems to support their quality improvement initiatives.

In 2009, Quality/Equality was one of RWJF's targeted program portfolios. RWJF is committed to improving the quality of care for all Americans.

RWJF's Quality/Equality approach had four major components, each representing significant investments and multiple partners:

Aligning Forces for Quality. Aligning Forces for Quality is the core strategy of the Foundation's long-standing commitment to improve the quality of health care that Americans receive. Through this national initiative, we are working to lift the overall quality of health and health care in targeted communities across the country.

Transparency. While performance measurement and public reporting about quality have become more common, we need far greater collaboration at the federal and local levels to standardize measurement and reporting activities and to create measures that are more meaningful to patients, providers and others.

Measuring Progress. We are devoting a substantial portion of our portfolio to research, tracking and evaluation. Some of this work will involve:

  • Using our existing research investments to assess more purposefully progress in the communities engaged in Aligning Forces for Quality.
  • Issuing targeted solicitations to the field to garner ideas for new interventions and tools to help spur the pace of quality change and transformation.

Communications. We are going to design and execute communications activities and support at multiple levels:

  • Local communities, for example, might require different types of targeted assistance with messaging, advocacy and engagement.
  • At the national level, we will actively share the stories and lessons learned from our regional work.

 Back to the Table of Contents


THE PROJECT

Under this grant, researchers from the National Quality Forum (NQF) identified topics for quality measures for mammography centers as the first phase of a multiphase study to further develop and test the widely held assumption that consumers will use information about the quality and safety of providers to select better care, thus stimulating providers to compete by improving quality.

Project staff convened a 12-member Mammography Coordinating Committee, composed of representatives from research institutions, clinicians, government agencies and consumer groups (see the Appendix for a list of members), to provide project oversight and guidance for the first phase of this project.

Subsequent to the completion of these tasks, the project was intended to move sequentially to phases in which other organizations would develop performance measures, perform pilot testing and conduct public education campaigns and evaluations, leading ultimately to the NQF's endorsement of the measures if they proved effective.

Measure Development

After conducting an extensive literature search for each of the topic areas related to mammography center quality that had been previously identified by the Mammography Coordinating Committee, the project team identified six preliminary measures for evaluation and discussion by the consumer focus groups. NQF convened a Coordinating Committee to identify potential domains for measures and to assess the scientific acceptability and feasibility of potential areas of measurement.

RWJF contracted a firm with expertise in consumer focus groups to conduct a total of 12 focus groups with six to 10 people each in Dallas, Los Angeles and Philadelphia. They selected these cities based on the number of mammography facilities, the number of mammograms performed annually, and the diversity of the female population using mammography services.

Participants were of varied educational levels, socioeconomic status and age and health status, and they had a range of different health insurance arrangements and experience with mammogram screening. Approximately 36 percent of the participants were ages 40 to 49, and 64 percent were age 50 and older.

The predominant themes that emerged from the focus group discussions are captured in the resulting composite statements:

  • Women with no mammogram experience expressed more interest in the technical quality of the mammography center (e.g., false-negative and false-positive rates or the education/experience of the mammographer) than did those who already had had one or more mammograms.
  • A majority of participants believed that primary care providers have (or should have) the data on the technical quality of a mammography center and should decide to which center to refer their patients based on those data.
  • "A mammogram is a mammogram is a mammogram." That is, focus group participants generally had the perception that mammograms are like routine blood tests and that there is little variation in quality.
  • Participants considered the experience of patients at mammography centers to be the most important information for selecting a mammography center, since from their perspective there is little, if any, variation in the technical aspects of mammography.
  • Participants believed that their choice among centers was limited to the primary care provider's hospital, health plan or health clinic affiliation. However, most of the women acknowledged that they did not know the extent of their choices and had not considered exploring options beyond those given by their primary care provider.

Based on the focus group information, committee expertise and evaluations of the recommended areas for measurement, the committee then recommended six potential measures for further development and pilot testing. The team also provided possible public educational opportunities and recommendations for each of the potential measures.

 Back to the Table of Contents


RESULTS

The project team reported the following results in its project report, Consumer-Focused Measures of Mammography Center Quality: Coordinating Committee Report:

  • The project team developed six measures for defining quality mammography centers:
    • Certification status: whether a mammography center possesses a certificate from the U.S. Food and Drug Administration or a state.
    • Average turnaround time for reporting results (for both normal results and results requiring more study).
    • Patient experience with care, based on standardized satisfaction surveys conducted at a point of service. Categories would include:
      • Treatment by staff
      • Accessibility and accommodation for women with special needs
      • Timeliness
      • Maintenance of privacy
      • Diversity of patient demographics
    • False-positive rates.
    • False-negative rates.
    • Percentage of screening mammograms performed, compared to all radiological procedures performed or as a percentage of all mammograms performed (screening and diagnostic) if at a comprehensive breast clinic.

 Back to the Table of Contents


CONCLUSIONS

Based on the focus group discussions and input from the Mammography Coordinating Committee, the project team developed the following observations, as reported in Consumer-Focused Measures of Mammography Center Quality: Coordinating Committee Report:

  • There is a need for more heightened awareness among consumers and primary care providers on the variation in quality of prevention/screening procedures. The focus group perceptions that "a test is a test" and that regulation ensures that there is no variation in technical quality suggest that a critical piece of creating effective quality measures for mammography centers is a public education campaign to help consumers more accurately gauge quality.
  • In order to study how performance measurement data might change the marketplace, it will be important to include a diverse group of women, including those with disabilities, in the discussion on consumer use of performance measures. Accessibility to mammography machines, staff awareness of people with special needs and the ability to accommodate these needs are important for the technical and service quality of a mammography center.
  • The potential measures identified in this project may require a level of data retrieval and coordination that is not consistently available across local, state or regional areas. To calculate rates on technical quality accurately, access to pathology reports is critical. This project highlights the importance of registries and tracking systems to facilitate the data retrieval needed to continually improve the quality not only of screening mammography but also of cancer care in general.

 Back to the Table of Contents


LESSONS LEARNED

  1. Develop mammography center quality measures for primary care providers as well as consumers. Even though primary care providers are inundated with increasing amounts of data, primary care provider organizations indicated in informal conversations that some of the information considered in this project would help them make a better referral fit for their patients. (Project Director/Francis)

 Back to the Table of Contents


AFTER THE GRANT

The project team determined that the hypothesis that this project was designed to test-that consumers will act on information about provider quality to select better care-did not fully resonate with consumers because there was confidence among the consumers interviewed that quality of mammography was not a pervasive issue. However, there was evidence that performance measures related to the experience of care for those with disabilities was an important issue. Overall, the measurement areas identified through this project did not fully match what consumers in focus groups said was important to them in making decisions about mammography.

"Although feasibility testing could have advanced after measure development, one might reasonably predict that making information available on mammography center performance would not significantly influence consumer decision-making unless an a priori public education campaign was undertaken," the project director said.

 Back to the Table of Contents


GRANT DETAILS & CONTACT INFORMATION

Project

Study to determine standardized measures that define quality mammography centers

Grantee

National Quality Forum (Washington,  DC)

  • Amount: $ 187,613
    Dates: November 2002 to October 2003
    ID#:  047154

Contact

Merilyn D. Francis, B.S.N., M.P.P.
(703) 983-5113
mfrancis@mitre.org
Kenneth W. Kizer, M.D., M.P.H.
(949) 369-4104
kennethwkizer@aol.com

 Back to the Table of Contents


APPENDICES


Appendix 1

(Current as of the time of the grant; provided by the grantee organization; not verified by RWJF.)

Mammography Coordinating Committee

Diane Blum, M.S.W. (Co-Chair)
Executive Director
Cancer Care
New York, N.Y.

Valerie Jackson, M.D. (Co-Chair)
Chair
Department of Radiology
Indiana University School of Medicine
Indianapolis, Ind.

Zora Brown
Director of Health and Cultural Affairs
INTEGRIS Health
Oklahoma City, Okla.

Joyce Dubow, M.A.
Associate Director
AARP Public Policy Institute
American Association of Retired Persons
Washington, D.C.

Charles Finder, M.D.
Center for Devices and Radiological Health
Department of Health and Human Services
U.S. Food and Drug Administration
Rockville, Md.

Rita W. Heinlein, R.T.
Mammography Consulting and Educational Services
Clarksville, Md.

M. Carolina Hinestrosa, M.A., M.P.H.
Executive Vice President of Programs and Planning
National Breast Cancer Coalition Fund
Washington, D.C.

Cynthia Lasko
Practice Administrator
Clinical Radiologists Medical Imaging, PA
Silver Spring, Md.

Susan E. Kutner, M.D.
Chair
Kaiser Permanente Northern California Breast Cancer Task Force
Santa Teresa Kaiser Hospital
San Jose, Calif.

Donna Marshall, M.B.A.
Executive Director
Colorado Business Group on Health
Boulder, Colo.

Judith Panko Reis, M.A., M.S.
Director
Rehabilitation Institute of Chicago
Health Resource Center for Women with Disabilities
Chicago, Ill.

Alice S. Petrulis, M.D., F.A.C.P.
Chief Medical Officer
Ohio KePRO
Seven Hills, Ohio

 Back to the Table of Contents


Report prepared by: Robert Crum
Reviewed by: Martha Fay
Reviewed by: Marian Bass
Program Officer: Lori A. Melichar

Most Requested