Leadership for Better Health: Evaluation of the RWJF Change Leadership Initiative
Program Period: March 15, 2015—March 31, 2025
Available Evaluation Reports (This section will be periodically updated with newly released reports.)
About the Change Leadership Initiative (CLI)
Today’s health challenges are influenced by factors well beyond the clinic walls, from our access to economic opportunity to the safety of our surroundings. Through our leadership development programs, the Robert Wood Johnson Foundation supports and connects change leaders nationwide who are working to build a Culture of Health. The CLI is made up for four Change Leadership Programs (CLP): Clinical Scholars (CS), Interdisciplinary Research Leaders (IRL), Culture of Health Leaders (COHL), and Health Policy Research Scholars (HPRS).
CLI Goals/Objectives
While the CLI as a whole aims to develop leaders who will help to build a Culture of Health for our nation, each CLP approaches this objective in its own way and is led by its own national program center (NPC).
- CS prepares and supports interdisciplinary teams of health care professionals to use their voice and skills in collaboratively tackling complex health issues in their communities.
- IRL equips teams of researchers and community partners to apply advanced leadership skills and rigorous research in meeting pressing community health needs.
- COHL supports leaders from all sectors and professions to create collaborative solutions that advance equity and move communities and organizations toward a Culture of Health.
- HPRS invests in scholars who are underrepresented in specific doctoral disciplines and/or are from historically marginalized backgrounds—scholars whose research and leadership can help shape policy toward a Culture of Health.
Evaluating the CLI
Evaluation has played a central learning role throughout the lifespan of the CLI and aims to help collaborators:
- gain more clarity on the intent of the programs;
- understand and respond to what is working and what can be improved in each of the programs;
- see the outcomes associated with program participation; and
- understand how the concept of a Culture of Health is being manifested in communities across the nation.
TCC Group has led the initiative-level evaluation since the programs’ inception in 2015. While each NPC has its own evaluator that assesses program-specific experiences and outcomes, TCC’s evaluation focuses on cross-initiative learnings based on experiences and outcomes tracked across all four programs. The evaluation team works alongside those setting strategy and those implementing the programs to administer strong data collection and infuse insights from the findings into how the programs are administered. Importantly, TCC works closely with the NPC evaluators to coordinate their evaluation activities in ways that minimize participant response burden, through data sharing, monthly meetings, and customized program-specific evaluation technical assistance and reports.
The evaluation provides RWJF with key insights about supporting leaders and leadership development programs. It consists of three main components, focusing both on cross-program learnings and on individual program findings:
1. Rapid Cycle Learning: TCC facilitates continuous learning by analyzing data about the application process, CLI convenings, and other participant experiences.
2. Formative Evaluation: TCC collects ongoing program quality and participant satisfaction data through participant surveys, which are aimed to guide programmatic decision-making. An external evaluator also conducts periodic formative evaluations to assess the overall effectiveness of the programmatic infrastructure.
3. Outcomes Evaluation: Each year, TCC collects initiative- and program-level data focused on experiences and outcomes for both active program participants and alumni.
The evaluation is on-going, and the contents of this website will be periodically updated as new reports and findings are released.
Evaluation Methodology
The CLI evaluation employs a number of different data collection tools and mechanisms, which are summarized in Table 1.
Register for upcoming webinars, where the TCC team will share live findings from the evaluation. Topics and a tentative schedule are listed below. (Recordings of previously recorded webinars will be posted when they become available.)
Webinars and Webinar Recordings
This interactive webinar focused on CLI participant data, summarizing participant experiences during the 2019-2020 program year. It begins with a discussion of the initiative as a whole (sharing data across the four programs) and is followed by break-out room presentations and discussions for each program (Clinical Scholars, Culture of Health Leaders, Health Policy Research Scholars, or Interdisciplinary Research Leaders). The recording only includes the large-group presentation and excludes the program-specific discussions. Watch the webinar recording.
TCC Group shares findings related to leadership program quality, expectations, and participant satisfaction. It discusses what works and for whom in leadership development, and how programs can adapt in real-time to meet changing needs. The webinar is informed by five years of data from four different national leadership development programs that comprise the CLI. Watch the webinar recording.
This webinar is the first of two presentations on theories of change and evaluation approaches for leadership development programs. Part 1 focuses on understanding intent. The foundation of evaluation is understanding the evaluand—what it is that you are evaluating. This can be tricky in leadership development programs and in this one-hour webinar, TCC Group explores various ways to think about what the purpose of your leadership development program is and how that can evolve over time. The discussion will also explore the potential unique role of foundations beyond funding. Join the TCC team as they share insights and invite discussion about what’s important, why, and how to know if leadership development programs are working! Watch the webinar recording.
In Part 2 of this two-part presentation on theories of change and evaluation approaches for leadership development programs, TCC Group focuses on indicators and evaluation methods to assess effectiveness of leadership development programs. Once a leadership program has a clear purpose or intent (see Part 1 of the series), a strong evaluation plan needs to be in place to assess effectiveness in achieving the desired outcomes. But what is not always considered, is how to create an evaluation approach that allows for program adaptation and responsiveness to changes in the environment in which the leaders are operating. In this webinar TCC will share experiences in developing and implementing this type of evaluation infrastructure for four different national leadership development programs that comprise the CLI. Watch the webinar recording.
This interactive webinar focused on CLI participant data, summarizing participant experiences during the 2020-2021 program year. Watch the webinar recording.
This webinar focuses on what leadership in action looks like across the CLI programs and explores the different ways in which exercising leadership can look different based on an individual’s own demographics, their time in a leadership program, or post-program.
This webinar summarizes how leaders describe leadership work and how leadership activities might change based on context, individual identity, and personal development. Insights are based on multiple years of data from the CLI evaluation. The webinar is aimed towards those implementing, supporting, or participating in leadership development and/or health equity work. Participants will walk away with clear findings on how individuals taking the lead on health equity efforts across the country have defined leadership and how that may differ based on individual characteristics. Watch the webinar recording.
Leadership development evaluation often skips to the perceived good stuff—what happened as a result of the program? Given the often-rigorous effort of recruiting and selecting participants, this neglects a critical aspect of any program: the participants. This webinar will share insights about the recruitment process and applicant experience for leadership development programs. Watch the webinar recording.
This interactive webinar focused on CLI participant data, summarizing participant experiences during the 2021-2022 program year. Participation for this webinar was open only to CLI current participants, alumni, and program staff.
Lessons Learned to Date
Findings from the evaluation, a sampling below, are summarized into a
wide range of reports, links for which can be found in the sidebar.
Goal: In order to build a Culture of Health, the CLI seeks to expand its support to individuals who are new to the Robert Wood Johnson Foundation.
- Most applicants have had no prior experience with RWJF—either through funding or other types of relationships.
Goal: Building a Culture of Health requires diverse leaders, representing different ethnic/racial backgrounds, gender identities, and sectors. The CLI aims to recruit candidates that reflect this diversity.
- Applicants are increasingly coming from a more diverse pool of disciplines having increased from 18 to 28 different disciplines represented over the years.
- Across the programs, applicants continue to be more likely to be female and either African American or Caucasian with some differences across programs.
Goal: The CLI seeks to be accessible and offer a valuable experience for a diverse pool of participants.
- The CLI program appears to have broad appeal across a spectrum of participants. Statistical analyses revealed no significant differences in satisfaction among participants based on gender identities and expressions, racial and ethnic backgrounds, geographic locations, career levels, and sectors.
Goal: CLI program implementers seek to continually understand the needs of its leader participants so that the programs can be adapted accordingly.
- Across the four programs, participants have found their programmatic relationships and networking opportunities to be the most valuable facets of their programs. There is no consensus on the least valuable aspect of the programs.
- During 2020, the loss of connection was overwhelmingly cited by participants as the greatest loss of their program experience due to the COVID-19 pandemic. This was due to the absence of in-person meetings with their program cohort and the cross-program convenings.
Goal: The CLI aims to help build and expand the networks of those participating in the programs.
- Network strength tends to decline slightly while in the program and participants tend to not expand the sectors and disciplines within their network while in the program. This implies that the program does not appear to be significantly helping participants diversify their networks.
Goal: CLI participants will be actively involved in creating a Culture of Health through a variety of roles and actions.
- CLI participants were most frequently involved in thought leadership through training activities, advocacy efforts, and some journal publications.
- CLI participants are actively involved in creating policy change particularly at the organizational and state levels.
- CLI participants are very active in exercising leadership by serving as an advisor on a Board or committee, or by creating change within their own organizations. An Equity, Diversity, and Inclusion (EDI) lens is frequently applied to their work.
Goal: CLI alumni will continue to actively work towards a Culture of Health through a variety of roles and actions.
- Across the CLI, the most common ways that alumni exercise leadership are in applying Equity, Diversity, and Inclusion (EDI) principles, and in advising other people, organizations, or communities on Culture of Health-related concepts. Nearly two-thirds of alumni were recognized for their leadership by being asked to join steering or advisory committees.
- The leadership roles taken by alumni changed between their time as a participant and that as an alum; although much of this is fueled by the COVID-19 pandemic and the increasing momentum of the racial justice movement. Examples of applying EDI principles doubled during this timeframe and examples related to securing funding for community initiatives and adapting clinical practice also greatly increased.
Goal: Due to the diversity of the CLI participant pool, CLI alumni will spread Culture of Health concepts in a wide variety of formats and with various audiences.
- CLI alumni most frequently disseminated Culture of Health concepts by conducting trainings and engaging in advocacy. Almost all CLI alumni (91%) reported engaging in some form of thought leadership activity. The majority conducted trainings (71%) and advocacy efforts (63%) in the past year.
- Community/Network partner interviewees reported that alumni had a variety of important roles in their respective communities, including leading community efforts (63%), being a subject matter expert (58%), and being an advocate for change (53%).
Goal: CLI alumni networks will continue to grow and strengthen after they leave the program.
- The strength of CLI alumni’s networks generally increased slightly when comparing post-program to program baseline. However, the diversity of the networks remained about the same. Network connections tended to align with the sectors in which the participants did their Culture of Health work.
Goal: CLI program implementers seek to continually understand what aspects of the program have been most helpful for leaders as they continue their leadership journey and to identify any new areas of support needed.
- CLI alumni found the development of leadership skills and the CLI network to be the most helpful aspects of their program for their current leadership role(s). CLI alumni frequently credited their program with helping them develop an equity lens
Related Videos
CLI Evaluation Design and Dissemination
A series of brief videos provides an overview of the collaborative approach the evaluation team took in designing the CLI evaluation, including objectives, methodology, and dissemination.