The Program Being Evaluated
Achieving Competence Today (ACT) was a program of Partnerships for Quality Education, a national initiative of the Robert Wood Johnson Foundation (RWJF). ACT tested the use of a Web-based curriculum by medical residents and graduate nursing students at more than 20 schools as a means for them to identify problems in patient care quality and collaboratively use systems-based approaches to improving care. RWJF funded ACT from May 2003 through September 2008.
About the Evaluation
In 2005, RWJF funded evaluators Michael Yedidia, Ph.D., and Colleen Gillespie, Ph.D., of Rutgers University to carry out an evaluation of ACT through a post-training survey of ACT II learners, and pre- and post-training study of ACT III learners and leaders.
The evaluators sought answers in four areas:
- How did learners assess the value of participating in the quality improvement process in the context of their training?
- What was the nature of the quality improvement projects conducted as part of ACT training? How were the projects valued by learners and leaders? How did they assess the contribution of ACT to quality improvement at their institutions?
- To what extent did learners collaborate with peers or staff from different disciplines in developing improvement projects? How did learners and leaders assess the quality of the collaborative experience? What were the factors that either facilitated or impeded effective interdisciplinary collaboration?
- What factors appeared to differentiate those health professions education programs prepared to pay $10,000 to continue participation in ACT from those that didn't?
Summary of Methods
To carry out the evaluation, Yedidia and Gillespie conducted:
- A post-training survey of the ACT II cohort of learners to elicit their assessments of the ACT training and their experience with and evaluation of quality improvement projects.
- A survey of leaders of the ACT initiatives to assess their valuation of the learning experience; their appraisal of the contribution of the projects to quality improvement at their institutions; and their estimation of the centrality of interprofessional collaboration to the ACT initiatives.
- A pre-training and post-training survey of ACT III learners and a survey of ACT III leaders (faculty preceptors, medical education or quality improvement directors) to yield data for analyzing determinants of their decisions as to whether or not to continue participation with no funding.
Knowledge and Impact
In a report on ACT submitted to RWJF in January 2007, the evaluation team reported the following:
- Learners (medical residents or graduate nursing students) commended the ACT training overall. Some 80 to 89 percent of ACT learners "strongly agreed" or "somewhat agreed" that they would recommend the initiative to others.
- Among key elements of ACT training, learners most highly rated the development of a quality improvement project. Content areas most frequently chosen by learners for their projects were:
- Effectiveness of care—providing appropriate care to those who could benefit and avoiding unnecessary care.
- Timeliness—reducing patient wait times and potentially harmful delays.
- Patient safety.
- ACT leaders (medical education or quality improvement directors, ACT preceptors and teachers) concurred with learners in their assessment of the value of learners' quality improvement projects to their institutions. More than 88 percent of leaders and learners felt they were of moderate or great value.
- Interdisciplinary collaboration was viewed, generally, as very important to the work of quality improvement, with 65 percent of leaders and 69 percent of learners holding this view. Learners, regardless of discipline, credited their ACT training with improving their outlook on two key dimensions of interdisciplinary work:
- Developing trust among collaborators.
- Becoming more aware of the limits of a single-disciplinary approach to care.
- When asked what most facilitated collaboration, learners' most prominent response was "the necessity of getting the job done."
Join the Commission on June 19, 2013 for a public meeting to raise awareness of how non-medical factors influence health and move public- an...
This is the agenda for the June 19, 2013 RWJF Commission to Build a Healthier America public meeting.
Scheduled speakers for the June 19, 2013 RWJF Commission to Build a Healthier America public meeting.
Learn how to improve care transitions and prevent avoidable hospital readmissions, and pick up nursing and medical education con-ed credits.
A wide variety of resources have been developed for the RWJF Commission to Build a Healthier America. Explore them all to learn more about w...
"We often see the benefits of diversity as being for minorities," Angela Amar writes. "We seldom see that the majority benefits as well."
A White House “Champions of Change” event yesterday honored 12 men and women who spent their careers improving the lives of children who hav...
The full list of commissioners for the re-convened Commission to Build a Healthier America, led by Mark McClellan and Alice Rivlin.
A short distance can mean large disparities in health. Across America, people living just a few miles apart have dramatic differences in lif...
How the County Health Rankings helped US News size up children's hospitals.
The Robert Wood Johnson Foundation is working to increase awareness and understanding of the impact of ACEs and the need to develop effectiv...
We create new opportunities for better health by investing in health where it starts—in our homes, schools, and jobs.