February 2007

Grant Results

SUMMARY

In 2002, a research team surveyed state Medicaid officials on state efforts to measure and improve the quality of care delivered to beneficiaries in Medicaid managed care programs.

Specifically, the team sought to assess the states' use of value-based health care purchasing — the term for a broad array of efforts aimed at taking quality as well as cost into account when procuring health care services.

The team, led by Arnold M. Epstein, M.D., M.A., a professor at the Harvard University School of Public Health, compared the 2002 survey data with data from two similar surveys conducted in 1996 and 1999.

Key Findings
In 2004, the team reported its findings in two articles in Health Affairs. One article focused on state oversight of health maintenance organizations (HMOs), the other on primary care case management programs in Medicaid.

  • The 2002 survey found a "modest" increase in the percentage of state Medicaid managed care programs collecting performance data from health plans on patient satisfaction, patient access to care and the quality of care.
  • More Medicaid agencies were feeding health plan performance data back to the health plans, but the states still "rarely" provided the information directly to Medicaid beneficiaries.
  • There was an increase in the number of Medicaid agencies reporting that at least one health plan had documented improvement in at least one performance measure.
  • State Medicaid agencies were more likely to collect performance data from HMOs than from primary care case management programs — programs under which a Medicaid agency contracts directly with physicians to provide patient care.

Funding
The Robert Wood Johnson Foundation (RWJF) supported this project from March 2001 through November 2005 with three solicited grants totaling $820,243.

 See Grant Detail & Contact Information
 Back to the Table of Contents


THE PROBLEM

In the 1990s, the states increasingly converted their Medicaid fee-for-service coverage to managed care as a means of controlling health care costs.

By 1998, 90 percent of the states had Medicaid managed care programs in place, and more than half of the nation's Medicaid beneficiaries were enrolled in managed care programs, according to the federal Centers for Medicare & Medicaid Services (CMS).

With managed care, the role of state Medicaid agencies changed from primarily handling and paying claims to selecting and negotiating with health plans.

Managed care health plans receive a fixed amount of money per enrollee to cover a defined set of services over a specified period regardless of actual services provided. This arrangement, termed capitation, can be an incentive to minimize the amount spent on care.

As a result, the growth in Medicaid managed care spurred concerns about the quality of care provided to Medicaid beneficiaries.

Value-Based Health Care Purchasing

To determine how state Medicaid agencies were adapting to managed care — especially in implementing quality management practices — researchers led by Arnold M. Epstein, M.D., M.A., at the Harvard University School of Public Health surveyed state Medicaid officials across the nation in 1996 and again in 1999. Epstein is a professor and chair of the school's health policy and management department.

The two surveys sought to assess the agencies' progress toward value-based health care purchasing, the term for a broad array of efforts to take quality as well as cost into account when procuring health care services. Value-based health care purchasing strategies include:

  • Obtaining data on patient satisfaction, patient access to care and the quality of care to help assess and manage purchasing decisions.
  • Offering incentives to health care plans and providers to encourage best practices and reward improved health status of patients.
  • Providing information and education to patients to help them become more effective health care consumers and active participants in maintaining their health.

The 1996 survey, which was supported by the Commonwealth Fund, found that state Medicaid agencies were beginning to mandate quality measurement and reporting but were providing little comparative feedback to health plans or to plan enrollees for use in selecting a health plan.

By 1999, most states were measuring the quality of care and sharing the information with the health plans. However, the researchers reported that "few if any states" could show evidence of actual quality improvement by a health plan.

This survey was also supported by the Commonwealth Fund. In addition, one of the researchers, Haiden A. Huskamp, Ph.D., received support from the Center for Health Care Strategies, under its Medicaid Managed Care Program, funded by RWJF (ID# 027327).

Findings from both surveys appeared in the February 2002 issue (volume 28, issue 2) of the Journal of the Joint Commission on Accreditation of Healthcare Organizations. The article is titled "The Evolution of Quality Management in State Medicaid Agencies: A National Survey of States With Comprehensive Managed Care Programs," by B.E. Landon, H.A. Huskamp, C. Tobias and A.M Epstein.

An Interest in New Data

In 2001, Epstein and his colleagues again wanted to survey state Medicaid agencies on their quality management practices. This time, in addition to health plans, the team was interested in learning about the states' quality management of primary care case management programs.

Primary care case management programs are a form of managed care but distinct from capitated health plans. In these programs, the Medicaid agency itself acts like a health plan, contracting directly with physicians to provide and coordinate patient care.

The use of primary care case management programs was growing, especially in states where commercial health plans were abandoning Medicaid participation, according to the researchers. However, little was known about how Medicaid agencies were monitoring the quality of these programs.

In addition to the Medicaid market, Epstein and his colleagues wanted to study the role of value-based purchasing practices in the private sector, which also had experienced significant growth in managed care.

By 2000, more than 70 million Americans with employer-sponsored health insurance were enrolled in health maintenance organizations (HMOs) and even more were enrolled in preferred provider organizations (PPOs) and other forms of managed care, according to the research team.

 Back to the Table of Contents


RWJF STRATEGY

RWJF staff believed the results of surveys of purchasing practices might be useful in promoting the value-based approach and informing the work of two RWJF national programs:

In conjunction with this strategy, in 2001 RWJF supported a survey of members of the National Association of Manufacturers on their experience with value-based health care purchasing. (See the Grant Results on ID# 042103.)

 Back to the Table of Contents


THE PROJECT

The research team led by Epstein surveyed state Medicaid officials in 2002 on state efforts to measure and improve the quality of care delivered to beneficiaries in Medicaid managed care programs. Specifically, the team sought to assess the states' use of value-based health care purchasing.

The team compared the 2002 survey data with data from the 1996 and 1999 Medicaid surveys and reported its findings in Health Affairs in 2004.

RWJF supported the work from March 2001 through November 2005 with three solicited grants totaling $820,243 awarded to the Harvard University School of Public Health (ID#s 040657, 043137 and 043782).

Survey Development

Under the first two grants (ID#s 040657 and 043137), the researchers developed the survey questionnaire, tested it in four states and made revisions. To assist in that process as well as to conduct the survey, the team hired the Health and Disability Working Group, a research, technical assistance and training collaborative at the Boston University School of Public Health.

Staff members from RWJF's Medicaid Managed Care Program and National Health Care Purchasing Institute provided input as well.

The survey included a core set of questions on managed care health plans adapted from the 1999 questionnaire. These questions sought information on:

  • The state's Medicaid managed care activities, including the number of years managed care had been in place, the number of contracting health plans and the proportion of Medicaid beneficiaries enrolled.
  • The agency's efforts to collect performance measures of patient satisfaction with care, access to care and quality of care and to feed the data back to the health plans and enrollees.

    For example, as a measure of access to care, the survey asked if the state agency collected and fed back data on the length of time that patients were kept waiting in a provider's office for an appointment.

    The several quality-of-care measures in the survey included childhood immunization rates and data on the testing of diabetics for blood-sugar control.
  • Incentives, penalties or other interventions to improve health plan performance. For example, the survey asked if the agency provided financial bonuses for health plans that achieved certain quality measures or imposed penalties on those that failed to achieve specified measures.

In addition to questions from the previous survey, the 2002 questionnaire included two entirely new sets of questions. One focused on quality oversight of primary care case management programs. The other, titled "viewpoints," sought to assess the importance that agency officials placed on various value-based purchasing strategies.

The 2002 survey — as did the 1999 and 1996 surveys — asked for information from the previous year. Thus the three surveys provided comparison data for 2001, 1998 and 1995.

Survey Implementation and Analysis

Under the third grant (ID# 043782), staff of the Health and Disability Working Group, a subcontractor, conducted structured telephone interviews April through September 2002 with directors and quality managers of 47 Medicaid agencies (46 states plus the District of Columbia) that had Medicaid managed care programs in place as of July 1, 2001.

The four non-participating states were Alaska and Wyoming, which did not have Medicaid managed care programs, and Nebraska and South Dakota, which did not respond to the survey.

Of the 47 participating agencies:

  • Thirty-eight had an HMO program, including 22 with only an HMO program and 16 with both an HMO program and a primary care case management program.
  • Twenty-five had a primary care case management program that met the study's definition for such programs, including nine with only a primary care case management program.

As part of their analysis of the survey results, the researchers examined the association between Medicaid agency value-based purchasing and health plan quality of care. The team used two tools in this analysis:

A Change in Project Plans

The research team had planned also to develop and implement surveys to assess value-based health care purchasing by private sector employers and health plans. The researchers developed the private purchaser survey and began work on the health plan survey.

However, RWJF was unable to fund further work on these two surveys because it cut back new grantmaking. At this time (2002–2003), RWJF's corpus had stopped growing due to a downturn in the stock market, and commitments to existing programs and projects used up available funds.

Communications

The research team published two articles on the Medicaid survey in the journal Health Affairs:

See the Bibliography for the full citations.

 Back to the Table of Contents


FINDINGS

Quality Management in Medicaid HMO Programs

The research team reported the following in the Health Affairs article "The Evolution of Quality Management in Medicaid Managed Care":

  • In the period 1995–2001, the number of states with Medicaid managed care programs, especially programs enrolling the disabled, initially increased but then declined.
    • Between 1998 and 2001, the number of states with managed care programs for the disabled fell from 31 to 22 while the number with plans enrolling beneficiaries of Temporary Assistance for Needy Families (TANF), the federal welfare-to-work program, fell from 45 to 39 (including Nebraska, a state with a program that did not respond to the survey).
  • The average number of health plans participating in Medicaid managed care programs also decreased.
    • States reported an average of 10.5 health plans serving the TANF population in 1995 and 8.4 health plans in 1998. For 2001, states reported a mean of 6.4 plans serving that population.
  • Despite the decline in the number of Medicaid managed care programs, "there were modest increases in the proportion of Medicaid agencies collecting data [on most performance measures] between 1998 and 2001."
    • In 2001, 89 percent of the states were collecting information on patient satisfaction compared to 69 percent in 1998, a 29 percent increase.
    • The number of states collecting blood-sugar-control data on diabetic patients more than doubled — from 24 percent in 1998 to 58 percent in 2001.
  • However, in contrast, the number of states collecting performance data related to mental health or substance abuse was "relatively low".
    • In 2001, 32 percent of the states collected data on the percentage of mental health/substance abuse patients who received a follow-up visit within 30 days of hospital discharge — compared to 53 percent in 1998, a 39.6 percent decline.
  • The number of states releasing health plan performance data to the health plans increased from 1998 to 2001. However, most states still did not provide data directly to beneficiaries.
    • In 2001, 32 of the 38 states with HMO programs (84 percent) released patient satisfaction data to health plans while 14 states (37 percent) provided the information to beneficiaries.
    • In 1998, 30 of the 45 states that then had HMO programs (67 percent) released patient satisfaction data to health plans while six states (13 percent) provided the information to beneficiaries.
  • Between 1998 and 2001, in "most instances, there was little or no increase" in the number of states reporting that health plans had targeted specific areas of patient satisfaction, access to care or quality of care for improvement.
    • In 2001, 23 states (61 percent) reported that health plans had targeted patient satisfaction with specific plans for improvement compared to 20 states (44 percent) in 1998.
  • In 2001, more states reported documented improvement by at least one health plan in at least one performance measure.
    • In 2001, 17 states reported that at least one health plan had documented improvements in patient satisfaction, compared to 10 states in 1998 and four states in 1995.
  • In 2001, almost a third of the states with HMO programs (12 of the 38 states) included financial bonuses or penalties related to quality in their contracts with health plans. However, these provisions were "not commonly used." During 1999–2001, five states made use of penalty provisions while three states used bonuses.
  • In 2001, six states had non-financial bonuses in their health plan contracts and 14 had non-financial penalties. However, these provisions "also were rarely used."

Quality Oversight in Primary Care Case Management Programs

The research team reported the following in the Health Affairs article "Quality Oversight in Medicaid Primary Care Case Management Programs":

  • State agencies were "much less likely" to collect performance data from primary care case management programs than from HMO programs.
    • Of the states that had only a primary care case management program, 67 percent collected data on satisfaction with care, compared to 86 percent of states that had only an HMO program.
    • Of the states that had both programs, 63 percent collected this data for the primary care case management program, while 94 percent collected data on satisfaction with care for the HMO program.
  • State agencies more commonly collected data on satisfaction with primary care case management programs than on the programs' quality of clinical care.
    • Of the 25 states with primary care case management programs, 64 percent collected satisfaction data while 12 percent collected data on cervical cancer screening.
    • No state collected data on mental health measures.
  • Feedback of performance results to the public and providers was less frequent in primary care case management programs than in HMO programs.
    • Some 66 percent of the 38 states with HMO programs provided data on patient satisfaction to the public compared to 12 percent of the 25 states with primary care case management programs.
  • Primary care case management programs often did not use traditional quality assurance functions such as credentialing and selection or deselection of primary care physicians.
    • Some 68 percent of the programs verified the training, licensing and certification of primary care providers.
    • Some 60 percent reviewed the adequacy of the primary care provider's office hours.
    • Some 56 percent reviewed the primary care provider's malpractice history.
    • Some 32 percent conducted an on-site review of providers' offices.

Value-Based Health Care Purchasing and Medicaid Managed Care Health Plan Performance

In a January 9, 2006 report to RWJF, the researchers said:

  • "Contrary to our expectations, we found no statistically significant associations of VBP [value-based purchasing] with either baseline levels of performance or with change in performance over time."
  • While the result was "disappointing," they wrote, there could be several explanations, including:
    • Medicaid agencies might be more likely to pursue valued-based purchasing in clinical areas where performance is poor and improvement takes a long time.
    • The Medicaid managed care programs were conducted in an ineffective manner.
    • The number of states in the sample was too small statistically to detect a small but real effect.

Limitations

In the Health Affairs articles, the research team acknowledged several limitations on the survey findings, including:

  • The survey relied on self-reports about specific programs and activities of the state agencies. While encouraged to consult their colleagues if necessary, respondents may not have had accurate knowledge about all of the issues surveyed.
  • "It is also plausible that the activities reported reflect the process 'as designed' instead of 'as implemented,' and that these programs may be less effective than envisioned."

 Back to the Table of Contents


CONCLUSIONS

Researchers reported the following conclusions in Health Affairs:

  • The survey data on health plans "suggest that trends in quality management are going in the right direction, although there is certainly opportunity for many states to intensify their efforts."
  • "Despite growing enrollment, PCCM [primary care case management] programs appear less likely to use the quality oversight strategies employed by Medicaid health plans."

 Back to the Table of Contents


SIGNIFICANCE TO THE FIELD

The researchers wrote in Health Affairs that to their knowledge, the 2002 survey was the "first systematic assessment of quality measurement and management activities" nationally among primary care case management programs.

In a July 2006 interview for this report, Epstein suggested that the results showed the quality management of primary care case management programs was less robust than generally had been believed.

 Back to the Table of Contents


RESULTS

Staff of RWJF's Medicaid Managed Care Program used the survey findings to guide the program's work with states to improve Medicaid managed care quality and access. The data "showed us that there were trends in the right direction, but that we had a lot more work to do," said Nikki Highsmith, the program's deputy director.

As a result of the findings, the program staff:

  • Targeted technical assistance efforts to states that were not yet collecting and using performance data. The staff created tools to help these Medicaid agencies disseminate performance data and improve quality and provided assistance to Medicaid officials in one-on-one and group sessions.
  • Targeted assistance in quality improvement to states with primary care case management programs.

The National Health Care Purchasing Institute closed before the research team reported its findings.

 Back to the Table of Contents


LESSONS LEARNED

  1. A greater understanding of the term value-based health care purchasing would support efforts to improve quality of care in the Medicaid market. The researchers learned that the term value-based purchasing is ambiguous. They suggested that increasing knowledge of value-based purchasing, including barriers to its use and characteristics of health plan purchasers who use it, would help promote its adoption. (Project Director/Epstein)
  2. Be prepared to do extra work when using a database that was not intended for research purposes. In order to use the HEDIS database, which was not designed for research, the research team had to do a large amount of additional work. This included sorting out multiple submissions from health plans that reported separate data for different parts of their networks. For example, a plan operating in the Philadelphia area might submit data separately for Pennsylvania, New Jersey and Delaware. (Project Director/Epstein)

 Back to the Table of Contents


AFTER THE GRANT

With funding from the federal Agency for Healthcare Research and Quality, Epstein and his colleagues finalized and conducted the private purchaser and health plan surveys begun under this project.

In 2005, they surveyed more than 600 private purchasers and, separately, 252 HMOs. The team reported on the HMO survey in the November 2, 2006 issue of the New England Journal of Medicine (355[18]: 1895–1902).

The article, "Pay for Performance in Commercial HMOs," said that 126 health plans — just over half of the respondents — included incentives in their provider contracts. Of the 126, 90 percent had incentive programs for physicians, and 38 percent had incentive programs for hospitals.

The Washington Post covered the findings in a piece, "U.S. Lags in Several Areas of Health Care, Study Finds" on November 3, 2006.

 Back to the Table of Contents


GRANT DETAILS & CONTACT INFORMATION

Project

Studying Value-Based Health Care Purchasing

Grantee

Harvard University School of Public Health (Boston,  MA)

  • Amount: $ 100,000
    Dates: March 2001 to August 2001
    ID#:  040657

  • Amount: $ 31,442
    Dates: November 2001 to November 2001
    ID#:  043137

  • Amount: $ 688,801
    Dates: December 2001 to November 2005
    ID#:  043782

Contact

Arnold M. Epstein, M.D., M.A.
(617) 432-3415
aepstein@hsph.harvard.edu

 Back to the Table of Contents


APPENDICES


Appendix 1

Glossary

Capitation (capitated): Payment of a fixed amount per health plan enrollee to cover a defined set of services over a set time period.

Consumer Assessment of Healthcare Providers and Systems (CAHPS): A repository of survey data on patients' experiences with health care products and services maintained by the federal Agency for Healthcare Research and Quality.

Health and Disability Working Group: A research, technical assistance and training collaborative at the Boston University School of Public Health.

Health plan: An organization that provides health insurance for an enrolled population.

Health Plan Employer Data and Information Set (HEDIS): A set of standardized measures of health plan performance compiled by the National Committee for Quality Assurance.

Health maintenance organization (HMO): A managed care plan that integrates financing and delivery of a comprehensive set of health care services to an enrolled population.

Managed care: A general term for organizing doctors, hospitals and other providers into groups in order to enhance the quality and cost-effectiveness of health care. Managed care organizations include health maintenance organizations (HMOs) and preferred provider organizations (PPOs).

Medicaid Managed Care Program: An RWJF national program that works with state Medicaid agencies, health plans and consumer groups to improve the quality of-and access to-Medicaid managed care, especially for beneficiaries with chronic illnesses or disabilities.

National Health Care Purchasing Institute: An RWJF initiative to train health care purchasing executives from the public and private sectors to use their buying power to improve the quality and cost-effectiveness of care. The institute operated from 1999 to 2003.

Preferred provider organization (PPO): An arrangement designed to supply health care services at a discounted cost by providing incentives for members to use designated providers, who contract with the preferred provider organization at a discount.

Performance measures: A measure of how well a health plan does in providing health services to its enrolled population. A performance measure can be used as a measure of quality. Examples of measures include the percentage of diabetics receiving annual referrals for eye care, and the percentage of enrollees indicating satisfaction with their care.

Primary care case management programs: Managed care programs under which a Medicaid agency contracts directly with physicians to provide patient care.

Quality management: An organization-wide process of measuring and improving the quality of the health care provided by a managed care organization.

Quality measurement: Quality measurements typically focus on structures or processes of care that have a demonstrated relationship to positive health outcomes and are under the control of the health care system. The number of nurses per patient is an example. Another is the percentage of female patients of a specified age receiving mammography.

Temporary Assistance for Needy Families (TANF): The federal welfare-to-work program enacted in 1996 to replace the Aid to Families with Dependent Children (AFDC) program.

Value-based health care purchasing: The term for a broad array of efforts to take quality as well cost into account when procuring health care services.

 Back to the Table of Contents


BIBLIOGRAPHY

(Current as of date of this report; as provided by grantee organization; not verified by RWJF; items not available from RWJF.)

Articles

Landon BE, Schneider EC, Tobias C and Epstein AM. "The Evolution of Quality Management in Medicaid Managed Care." Health Affairs, 23(4): 245–254, 2004. (Abstract available at RWJF; full text available online.)

Schneider EC, Landon BE, Tobias C and Epstein AM. "Quality Oversight in Medicaid Primary Care Case Management Programs." Health Affairs, 23(6): 235–242, 2004. (Abstract available at RWJF; full text available online.)

Survey Instruments

"The Changing Patterns and Impact of Value-Based Purchasing: Survey of Medicaid Agencies." Boston University, fielded April to September 2002.

 Back to the Table of Contents


Report prepared by: Lori De Milto
Reviewed by: Michael H. Brown
Reviewed by: Molly McKaughan
Program Officer: Laura C. Leviton, Ph.D.

Most Requested