Evaluation of Partnerships for Quality Education

Evaluation conducted October 1999 through September 2004

The Program Being Evaluated

Partnerships for Quality Education (PQE) was launched in 1996 with an $8.3 million grant from The Pew Charitable Trusts. Guided by Pew's vision of better preparing primary care residents for practice in the evolving world of health care, PQE used its resources to foster collaboration between academic medical centers and managed care organizations. Between 1996 and 1999, PQE funded 66 partnerships to develop new curricula and new models for training future clinicians in the skills and competencies of managing care

In 1999, the Robert Wood Johnson Foundation (RWJF) awarded PQE a grant of $8.9 million to continue its work in developing new models for education. With funding from RWJF, PQE continued its work on the Partnerships Program, adapting the program as managed care changed, and added initiatives in teamwork (Collaborative Interdisciplinary Team Education) and chronic illness management (Take Care to Learn). In 2003, PQE launched the Achieving Competence Today (ACT) initiative to help residency and nurse practitioner programs transform teaching around systems and practice improvement.

About the Evaluation

Beginning in 1998—funded initially by the Pew Charitable Trusts, and then by RWJF—the evaluation consisted of a survey of 63 project directors in order to define the specific managed care competencies and related patient care tasks that residence program directors expected residents to learn as a result of the new training. (This initial work did not address itself to nurse practitioner students; however the findings informed the evaluation and curriculum design as it later applied to projects for both medical residents and nurse practitioners.)

Michael Yedidia, a professor at Rutgers University, and Jessica Greene, an assistant professor at the University of Oregon, Eugene, conducted the evaluation of the Partnerships Program.

Summary of Methods

For the evaluation of the Partnerships Program, Yedidia and co-evaluator Greene administered surveys that 56 of its 58 project directors completed in 1999 and 2000. They also developed a managed care knowledge assessment that project directors distributed to their learners in 2000; 600 learners at 17 of the projects completed it. The evaluation—surveys and the knowledge assessment—focused on their subjects before and after involvement with the Partnerships Program and measured the change in focus on managed care competencies among medical residency and nurse practitioner education programs and in their learners' knowledge of those competencies.

Knowledge and Impact

In 2000, Yedidia and his colleagues published an article in the Journal of the American Medical Association that identified the managed care competencies and associated clinical patient care tasks that both residency directors and managed care medical directors felt would be important to patient care in the next five years. See Program Results Report for additional information.

  • Using pre- and post-tests, evaluators found an increase in the numbers of managed care competencies addressed in training.
  • In a knowledge assessment of four broad managed care competency areas taken by 600 learners in 17 projects, overall scores improved by 8 percent between pre- and post-tests, while scores of a comparison group of learners decreased by 1 percent.
  • Project directors reported having strong buy-in for implementing their Partnerships curriculum from top-level administrators and reasonably strong interest in participating from learners. However, getting faculty to make the training a priority was a substantial challenge for more than half the projects. Approximately one-third of the project directors did not feel their faculty had sufficient expertise to teach the curriculum.
  • Project directors reported that the Partnerships Program made useful teaching materials available and that its national meetings were helpful for their project development. However, one-third of the project directors said that there was insufficient funding with which to develop a new curriculum. Three-quarters said it was difficult for them to devote the necessary time to making their Partnerships project a success.
  • Project directors appeared to have generally positive experiences working with their managed care partners. While the majority (76%) strongly or somewhat agreed that their partner was adequately involved in the development of the curriculum and with project implementation, 20 percent strongly disagreed.
  • The majority of projects (82%) reported that they planned to continue offering the managed care curriculum after Partnerships Program funding ended. Half said they planned to continue their partnership with their managed care organization—including 20 projects whose relationship began under Partnerships funding.