October 2004

Grant Results


From 2000 to 2002, staff of four organizations planned projects to demonstrate the use of financial incentives to reward providers for delivering high-quality health care. The organizations were:

Key Results
The planning grant initiative had the following outcomes:

  • Three of the four planning projects — those of DoctorQuality, the Institute for Health Policy Studies at the University of California, San Francisco, and the Rocky Mountain Health Foundation — produced a plan or concept for a pay-for-quality demonstration project.
  • Each of the three plans led to a proposal to the Robert Wood Johnson Foundation (RWJF) Rewarding Results national program for demonstration funding (for more information see Grant Results).

    The goals of Rewarding Results were to invent, prove and diffuse innovations in systems of provider payments and nonfinancial incentives that will encourage and reward high-quality care.
  • One of the three proposals — the one growing out of the DoctorQuality project — received implementation funding though Rewarding Results.

RWJF funded the planning with four separate grants totaling $257,261.

 See Grant Detail & Contact Information
 Back to the Table of Contents


The American health care system can make it financially difficult to provide quality care, especially in the treatment of chronic conditions. For example, to ensure first-rate care to a diabetic, the physician and his or her staff must maintain contact with the patient between office visits, monitor medications and track regular lab tests.

Few follow-up tasks of this type are reimbursable under most health care plans. Similarly, a hospital that institutes a disease-management program that seeks to shift patient care from inpatient to outpatient primary care runs the risk of reducing its own profitability.

This problem is especially acute in the federal Medicare program. "The payments are perverse. It pays us to harm patients, and it punishes us when we don't," Brent C. James, M.D., of Intermountain Health Care, a large health care system based in Salt Lake City, told the New York Times. The Times' December 5, 2003, article, "Hospitals Say They're Penalized by Medicare for Improving Care: Re-Examining Medicare," reported that Intermountain's quality initiatives, such as a system to ensure physicians prescribe the most effective antibiotic for pneumonia, had cost the company millions of dollars in lost admissions and lower Medicare reimbursements.

The Institute of Medicine recognized this troubling paradox in its March 2001 report Crossing the Quality Chasm, which detailed dysfunctional aspects of the American health care system. Among its recommendations, the report called for development and testing of payment policies that align compensation more closely with quality-improvement goals.

The concept of making quality care financially attractive has several shorthand names, including pay for quality, pay for performance and quality contracting. Whatever the wording, the terminology refers to models that align financial reimbursement with outcomes of care as measured by specific criteria, such as clinical process standards, patient satisfaction and quality of life.

Some employers and other purchasers of health care already have embraced this concept. The Leapfrog Group (a consortium of employers formed in 2000 to improve the quality of health care) includes in its purchasing principles the use of financial as well as non-financial incentives to reward provider efforts.

 Back to the Table of Contents


Improving the quality of care of people with chronic health conditions is an RWJF goal. Through several of its programs, including Improving Chronic Illness Care and Medicaid Managed Care Program, RWJF learned first hand that inadequate reimbursement can be an obstacle to attaining the goal.

In July 1998, Michael B. Rothman, M.P.P., joined RWJF as a senior program officer after holding health policy positions with the Colorado state government. Rothman, who had worked extensively in Colorado with purchasing-related issues, was anxious to spur increased use of the incentive concept.

"The reality is, we can't expect most providers to make major investments in learning how to improve quality and in hiring staff and making systemic changes if they are going to lose money doing it," Rothman later told a writer for the RWJF publication, Advances (Issue 1, 2004).

Prior to Rothman's arrival, RWJF had supported work in the related area of risk adjustment but had not been active in pay for quality. (Risk adjustment — which was a component of one of the four projects covered by this report — refers to the practice of increasing a health plan's compensation to cover its increased financial risk from enrolling people with conditions that are expensive to treat. Without risk adjustment, a health plan has a disincentive to provide quality care to high-cost patients.)

Rothman understood that some corporate purchasers had already begun building quality incentives into their health plan contracts. However, he believed that wide-scale progress was limited by several factors. One was the lack of established methodologies for determining the specific relationship between reimbursement and quality improvement: in short, how much incentive achieves how much improvement? Another was health plans' resistance to collecting clinically relevant performance indicators.

In 2000, Rothman got authorization from RWJF for $250,000 to fund as many as five pay-for-quality planning grants of up to $50,000 each. The grantees were to develop projects that would demonstrate efficient methods of linking compensation to quality. Although there was no firm commitment to funding a second round of grants, RWJF anticipated that this initial planning phase would lead to proposals for full-fledged demonstration projects costing from $500,000 to $1.5 million each to implement. Rothman tentatively planned to convene a panel of reviewers to evaluate the second-round proposals.

Instead of issuing a formal call for proposal for the planning grants, RWJF disseminated information about the opportunity through word of mouth and a two-page concept paper that Rothman distributed to organizations and individuals he knew to be interested in the pay-for-quality concept. The initiative was not a separate RWJF national program; the planning grants would be ad hoc awards.

Rothman, who had already identified several potential applicants through his own contacts, hoped that this expedited procedure would generate highly imaginative, ambitious proposals. RWJF's Clinical Care Management Team, which included Rothman, was to review and select the grantees.

The concept paper listed five elements to be included in each planning-phase project:

  • Identification of major health care purchasers and providers that could help plan a demonstration or possibly become a demonstration site.
  • Identification of one or more chronic diseases with care processes subject to measurement.
  • Design of provider incentives.
  • Selection of information systems to track care processes and performance.
  • Design of an evaluation approach that would produce results convincing to decision-makers at major purchasers and providers.

 Back to the Table of Contents


Information about the planning-grant opportunity did not circulate as widely as Rothman had expected. Too few people to whom he sent the concept paper forwarded it to other potential participants, Rothman said. He also was disappointed in the content of the proposals that RWJF did receive, saying they were more modest in scope and objective than had been hoped.

Nevertheless, from July 1, 2000, to September 1, 2001, RWJF awarded four pay-for-quality planning grants. Three of the four grantees were organizations that Rothman had identified as potential applicants. Two of the grants exceeded the $50,000 initially set as the limit.

ID# 039398 - DoctorQuality.com

DoctorQuality.com, Inc. (later shortened to DoctorQuality) received the first of the planning grants in July 2000. DoctorQuality was a new, for-profit company formed to provide Internet-based quality-management tools to the health care field. The company's founder and CEO, internist David J. Shulkin, M.D., previously served as chief medical officer and chief quality officer of the University of Pennsylvania Health System and was a 1990–91 Robert Wood Johnson Foundation Clinical Scholar. Company management included other former members of the University of Pennsylvania quality-management team. RWJF staff voiced initial concern about funding a for-profit entity to undertake activities that could lead to a marketable product. Company personnel gave assurance that all lessons learned from the project would be in the public domain.

The company had already developed the basics of a pay-for-quality model and used the four-month, $49,200 planning grant from RWJF (ID# 039398) to hone the plan and determine factors necessary to implement a demonstration. Key components of the company's planned pay-for-quality system included the following:

  • Health plan enrollees would complete a survey, either on paper or the Internet, providing baseline information on their health status, quality of life and other factors.
  • Each enrollee would also receive an online, disease-specific workbook that he or she would use to track health care visits and outcomes during the year.
  • At the end of the year, enrollees would complete their workbooks and a follow-up survey.
  • The physician and the provider system would be eligible for enhanced reimbursement based on patient satisfaction, quality-of-life scores and other outcome measures. The enrollees would be eligible for a rebate of up to $100 according to what extent their workbook and survey data showed they had maintained a healthy lifestyle and partnered with the physician to ensure quality care.

During the grant period, the team conducted a literature review of issues relevant to pay-for-quality contracting, including incentives to alter clinical behavior and strategies for the collection of outcome data. The project team also conducted group discussions and individual interviews with employers, health plan representatives and physicians to get reactions to the company's model and decide specifics of the incentive process.

As one key component of the planning process, DoctorQuality held a one-day planning meeting in Philadelphia in October 2000 with 15 invited representatives of purchasers and providers with potential interest in participating in a demonstration. The participants included representatives of the General Electric Co. and General Motors Corp.

ID# 039453 - Economic and Social Research Institute

The second planning grant went to the Economic and Social Research Institute, a nonprofit organization in Washington that conducts studies aimed at enhancing the effectiveness of social programs and health care services. RWJF has awarded a number of grants to the organization over the years. Mark W. Legnini, Dr.P.H., senior vice president, directed the planning project.

With support from Aventis Pharmaceuticals and the federal Agency for Healthcare Research and Quality, the institute had sponsored a meeting in Washington to explore the possibility of physicians and health care purchasers collaborating to improve health care quality through incentives. The participants in this January 2000 meeting included representatives of physician groups and several companies that purchased health care for their employees, including General Motors, LTV Corp. and Motorola. The application to RWJF was an outgrowth of that meeting.

The institute proposed to use the RWJF grant to work with purchasers and physician organizations to plan a quality-improvement program. The program would incorporate mutually agreed upon care processes and outcomes and experiment with different kinds of incentives that the purchasers would provide the physicians. Through the planning process, the project team hoped to identify one or more markets where the concept could be implemented as a pilot. Once the participating employers and providers were recruited, the institute viewed its role as evaluator of the implemented demonstration.

During the grant period, the project team focused on encouraging incentive projects in three locations. Institute staff conducted meetings and telephone conference calls with representatives of physician groups and employers in these communities and analyzed market data to project the number of likely participants in a pay-for-quality program. All three locations involved corporate participants from the January 2000 meeting:

  • Cleveland, where the project team worked with LTV and the Health Action Council, a coalition of businesses interested in measuring quality of care at area hospitals.
  • Austin, Texas, where Motorola was interested in initiating a program to manage chronic pain among the company's 1,200 local employees. Project personnel held discussions with Motorola's benefits director for North America and with the administrator of the Austin Regional Clinic, a multi-specialty group that provided physician services to Motorola's Austin employees.
  • Dayton, Ohio, where a coalition of providers and employers, including General Motors, was testing a new health care plan that used incentives to reward providers meeting performance targets set jointly by purchasers, practitioners, health system representatives and consumers.

ID# 032131 - University of California, San Francisco

The Institute for Health Policy Studies at the University of California, San Francisco received the third planning grant in August 2001. RWJF had anticipated funding the institute earlier but delayed the award until the organization completed a risk-adjustment project supported by a $2.48-million RWJF grant made in 1997.

This institute's earlier risk-adjustment project focused on developing an approach that would more accurately compensate health care plans for serving patients with high-cost chronic conditions, thus increasing patient access to quality care. The project's research team, led by Harold S. Luft, Ph.D., the institute director, and R. Adams Dudley, M.D., M.B.A., analyzed detailed health care data in an effort to relate cost of care to the specific severity of a patient's medical condition instead of to the general diagnosis for that condition, which is commonly the basis for cost calculations. (See Grant Results on ID# 029661.)

The pay-for-quality planning project built on this earlier work. Luft and Dudley proposed to develop a demonstration project that would couple their rate-adjustment concept with condition-specific measures for determining the quality of care delivered by hospitals. The measures would be more clinically detailed and cover a broader spectrum of care than possible with existing quality measurement systems.

By demonstrating the availability of credible data resources that permitted sophisticated risk adjustment and quality measurement, the team hoped to promote quality as a prime criterion that health plans can use to select hospitals. The university was concerned that some health plans in California were focusing on the prices that hospitals charge without considering the quality of care, an approach that penalized higher-cost institutions with strong reputations for quality. For example, one of the state's major health plans had begun excluding expensive hospitals from the plan's provider network, and another plan was requiring patients who used such hospitals to pay a higher copayment, the team reported.

Under the planning grant, Luft and Dudley worked to design a project that would focus on high-cost, chronic-care patients admitted to the hospitals at the five University of California medical schools (Davis, Irvine, Los Angeles, San Diego and San Francisco). The team held weekly discussions with leaders of the five medical centers to enlist their participation in the demonstration.

Through these meetings, the team identified the medical conditions to be studied and worked out methods of collecting and analyzing hospital patient data. Unlike some other pay-for-quality projects, the data resources were to include information abstracted from the hospitals' patient charts. The team also identified databases that collected detailed clinical information suitable for sophisticated rate adjustment and quality measurement and that were widely used and, thus, available in other markets that might adopt the model.

ID# 041555 - Rocky Mountain Health Foundation

RWJF awarded the fourth planning grant to Rocky Mountain Health Plans in Grand Junction, Colo. via its nonprofit affiliate, Rocky Mountain Health Foundation. Rocky Mountain Health Plans is a health maintenance organization (HMO) with 120,000 members statewide, 40 percent of them in Grand Junction and surrounding Mesa County. Rocky Mountain's leadership viewed development of a physician-incentive system as a necessary step to implementing a model of care for chronically ill patients served by the HMO's primary care physician network. (The HMO sought to implement the Chronic Care Model, which was developed by the RWJF national program Improving Chronic Illness Care; the model identifies essential elements of a high-quality chronic disease management system. (For more information on the Chronic Care Model go to www.improvingchroniccare.org/change.)

Under this grant, the project team designed a financial-incentive system to reward physicians for improved health outcomes among diabetic patients. In addition to holding planning meetings with Mesa County physicians, the team conducted a literature search on incentives and quality measurement, contacted authorities in the field, reviewed data on physician performance and outcome goals, and studied the results of a pilot project that used incentives in the management of depression.

Strategy Shift at RWJF

In the wake of the March 2001 release of the Crossing the Quality Chasm report, RWJF was anxious to stimulate additional innovative work in incentive-payment systems. In July of that year, Rothman began designing a second pay-for-quality initiative, this one to be a formal RWJF grant program administered by an outside organization and incorporating a structured site-selection process with a call for proposals and an advisory committee of experts.

In January 2002, the RWJF Board of Trustees authorized the new program — called Rewarding Results: Aligning Incentives with High-Quality Health Care — for $7.5 million over four years. The California HealthCare Foundation provided additional support. The program offered grants of up to $1 million, plus technical assistance, to employers, health plans and provider organizations to implement and evaluate new incentive systems. RWJF selected the Academy for Health Services Research and Health Policy (now known as AcademyHealth) in Washington to administer Rewarding Results as part of the National Health Care Purchasing Institute, an RWJF-funded resource for health care purchasers.

Although the new program was similar in concept to the earlier initiative, the eligibility criteria intentionally favored proposals involving a large number of enrollees, employees or patients (at least 75,000). RWJF hoped that the funded demonstrations, if successful, would influence the health care field, particularly the purchasing practices and policies of the Medicare program. RWJF wanted demonstrations of sufficient size to be viewed as valid operational tests.

With Rewarding Results in place, RWJF informed the four pay-for-quality planning grantees that they should seek demonstration-phase funding through this new national program, putting them in competition with other applicants.

 Back to the Table of Contents


In summary, the planning grant initiative had the following outcomes:

  • Three of the four planning projects — those of DoctorQuality, the Institute for Health Policy Studies at the University of California, San Francisco, and the Rocky Mountain Health Foundation — produced a plan or concept for a pay-for-quality demonstration project.
  • Each of the three plans led to a proposal to the Rewarding Results program for demonstration funding.
  • One of the three proposals — the one growing out of the DoctorQuality project — received implementation funding though Rewarding Results. However, DoctorQuality did not participate in the demonstration, which is ongoing.

A closer look at the results of each planning grant follows.

ID# 039398 - DoctorQuality

The DoctorQuality team outlined a three-year demonstration that would follow patients with one of several selected chronic conditions. A DoctorQuality software product populated with clinical information and other quality measures would calculate quality-based payments that would be made to individual providers. The participating provider system could also earn bonus payments based on improvement in aggregate patient satisfaction and quality-of-life scores as well as on achievement of targeted outcome measures. Small monetary and non-monetary incentives for patients were also possible.

Shortly after the planning grant ended, Maulik S. Joshi, Dr.P.H., a DoctorQuality cofounder and principal in the pay-for-quality project, contacted purchasers to determine their interest in testing the concept. The result was that a number of organizations worked together to develop further the details of a demonstration proposal that became the basis for an application by General Electric to Rewarding Results in 2002.

General Electric made cash as well as in-kind contributions to the effort to develop the demonstration project. According to Francois de Brantes, program leader for General Electric health care initiatives, the other collaborators included Partners HealthCare, an integrated delivery system based in Boston; Tufts Health Plan, a Massachusetts HMO; Medstat, a health care information company; the Lahey Clinic in the Boston area; the federal Centers for Medicare & Medicaid Services; and the National Committee for Quality Assurance, a nonprofit organization in Washington focused on improving health care quality.

The Rewarding Results proposal was one of seven selected for funding, and in January 2003 RWJF awarded General Electric $332,802 over three years (ID# 047848) to implement the project, which was named Bridges to Excellence.

A number of health plans, provider organizations and large employers (including Ford Motor Co., Procter & Gamble and United Parcel Service) are participating with General Electric in Bridges to Excellence. However, DoctorQuality, which was to provide electronic tools for the demonstration, ended up not participating. Joshi and Shulkin both left DoctorQuality in 2002, and subsequently the company sold its Web-based products and ceased operation.

ID# 039453 - Economic and Social Research Institute

The project team was unable to identify a suitable site for a pay-for-quality intervention and evaluation and, consequently, did not produce a plan for a demonstration. In some areas, the competitive nature of the health care market foreclosed the possibility of collaboration, the institute reported to RWJF. After a decade of dealing with HMOs and other commercial pressures, some physicians viewed collaboration with purchasers and insurers as economic coercion, the institute reported.

Even in communities where purchasers and providers had good relationships, the project team found that convincing employers to provide financial or other incentives for quality improvement could be "a tough sell." The institute report characterized a common response among employers as: "If we have to start paying extra for good quality medicine, what were we getting before?"

As a result of the failure to find a suitable demonstration site, the institute expended $31,854 of the initial $45,060 grant.

ID# 032131 - University of California, San Francisco

The planning team secured the participation of the five University of California academic medical centers in a plan to develop sophisticated measures of their quality of care and compare their performance with national and statewide benchmarks.

The five agreed to collect and share quality-of-care data with each other and with the university's human resources department, which handles health benefits for university employees. The plan called for the department to use the resulting data as the basis for deciding whether to get health plans to include the hospitals in their provider networks. The team believed the prospect of inclusion or exclusion would provide an incentive that would motivate the hospitals to excel. Given the university's status as one of the state's largest employers, the team expected the demonstration to influence other employers, providers and health plans.

The planned project targeted congestive heart failure, neonatal intensive care, adult intensive care, coronary artery bypass graft surgery and breast cancer, and identified measures and database tools for each condition. At each of the five hospitals, data collection would start with the review and abstraction of medical records and placement of the data in electronic databases. The five medical centers would pay the majority of the $2-million cost of data collection.

The team submitted the plan to Rewarding Results for funding but was not successful. Dudley said the proposed demonstration could not meet the program's size guidelines. The large hospital investment necessary to generate the in-depth quality data made it difficult to cover as many patients as the designers of Rewarding Results expected.

ID# 041555 - Rocky Mountain Health Foundation

The project team developed a plan for a demonstration that would test two hypotheses:

  • Linking significant financial incentives to key indicators of chronic-illness care will result in improved physician performance based on those indicators.
  • Combining financial incentives with implementation of a systematic approach to chronic illness management will result in physician performance that is significantly better than when providing no financial incentives or providing just incentives without a management model.

The plan called for tying additional physician reimbursement to improvement in the health of diabetic patients according to the percentage of patients who met goals for:

  1. glycemic (blood sugar) control
  2. improved glycemic control
  3. acceptable blood pressure
  4. acceptable cholesterol levels
  5. use of aspirin as recommended.

(Glycemic control is associated with decreased incidence of diabetic complications and other improved outcomes. Control of blood pressure and cholesterol and the use of aspirin also reduce the risk of complications, such as stroke and heart attack.)

The Mesa County Physicians Independent Practice Association agreed to fund a performance pool from which outcome-based incentive payments would be made. When the demonstration was fully implemented, physicians achieving the highest performance levels would receive additional pay equal to 5–11 percent of their base compensation under their HMO contract.

In 2002 the Rocky Mountain Health Foundation sought funding from Rewarding Results to implement a slightly modified version of this demonstration plan. The application was not successful. The proposed project did not cover as large a number of patients as the Rewarding Results program was designed to reach.


As part of their efforts to explore and build support for their demonstration plans, the project teams made presentations to various health care-related groups. However, they published no articles and disseminated no other materials on their initiatives.

 Back to the Table of Contents


The project teams reported the following lessons that may be helpful to other individuals or organizations interested in undertaking a pay-for-quality initiative:

  1. Pay for quality loses proponents when it moves from the drawing board to the pocketbook. Many employers are interested in the concept of providing financial incentives to improve the quality of care. But interest wanes when the discussion progresses to the real-world issue of paying the incentives. (ID# 039398 Project Team Member/Joshi)
  2. Cultural change in the medical profession may be more important than simply rearranging incentives. This project showed providers are resistant to quality-improvement initiatives. To reduce that resistance, there must be a cultural change that begins in medical school and residency, not with incentives and organizational restructuring for physicians already in practice. This project also demonstrated how few purchasers are willing and able to commit resources for health care services beyond paying what they consider to be the best price. (ID# 039453 Project Director)
  3. More sophisticated quality measures are necessary before the pay-for-quality approach can be fully effective. Pay for quality is not yet "ready for prime time," to use Dudley's phrase, and will not be until more basic research into quality measurement is conducted and better measurement tools developed. (ID# 032131 Project Team Member/Dudley)
  4. With adequate planning, it is possible to induce hospitals, even competitive ones, to join initiatives in quality measurement and risk adjustment. While hospitals generally disdain quality measurement and risk adjustment as providing data insufficient in detail, hospitals recognize that legitimate efforts can produce usable data. (ID# 032131 Project Director)
  5. When designing an incentive system, remember the old adage: "The devil is in the details." The concept of tying rewards to quality outcomes is simple, but creating an effective pay-for-quality system takes a significant amount of research, discussion and revision. (ID# 041555 Project Director)

 Back to the Table of Contents


General Electric's Bridges to Excellence demonstration began in April 2003 with a program for diabetic patients in three metropolitan areas (Cincinnati, Boston, and Louisville, Ky.) where the partner organizations have large numbers of employees, about 200,000 in total. The demonstration, which gives top-performing doctors bonuses of up to 10 percent of their regular income, issued its first payments in November 2003.

The project team at the University of California, San Francisco, ceased actively seeking funding for their proposed demonstration but did not give up on implementing the project eventually, Dudley said.

Rocky Mountain was moving ahead without RWJF funding to establish an incentive system for the care of diabetic patients as part of the HMO's effort to implement the Chronic Care Model.

 Back to the Table of Contents



Assessing the Feasibility of Pay-for-Quality Demonstration Projects


University of California, San Francisco, Institute for Health Policy Studies (San Francisco,  CA)

  • Amount: $ 99,907
    Dates: August 2001 to October 2002
    ID#:  032131


R. Adams Dudley, M.D., M.B.A.
(415) 476-8617
Harold S. Luft, Ph.D.
(415) 476-5147


DoctorQuality.com (Conshohocken,  PA)

  • Amount: $ 49,200
    Dates: July 2000 to October 2000
    ID#:  039398


David J. Shulkin, M.D.
(215) 707-7766


Economic and Social Research Institute (Washington,  DC)

  • Amount: $ 31,854
    Dates: September 2000 to March 2001
    ID#:  039453


Mark W. Legnini, Dr.P.H.
(202) 686-6981


Rocky Mountain Health Foundation (Grand Junction,  CO)

  • Amount: $ 76,300
    Dates: September 2001 to February 2002
    ID#:  041555


Lisa Fenton-Free
(970) 248-5027
Lori A. Stephenson, R.N.
(970) 248-5124

 Back to the Table of Contents

Report prepared by: Michael H. Brown
Reviewed by: Kelsey Menehan
Reviewed by: Marian Bass
Program Officer: Michael B. Rothman

Most Requested