May 2004

Grant Results

SUMMARY

From 1998 to 2003, the National Foundation for the Centers for Disease Control and Prevention (CDC Foundation) administered a pilot educational program to improve the management competency of public health professionals and the effectiveness of their agencies in Georgia, North Carolina, South Carolina and Virginia.

Key Results
The project achieved results in the following areas:

Key Findings
An evaluation of the Management Academy found the following:

  • Trained at a total cost of $2 million, graduates of the first three years of the academy successfully applied their skills and competencies to garner over $6 million in revenue for their agencies.
  • After graduating from the academy, the majority of participants used their management skills to implement their business plans in their local health departments.
  • Overall, participants were able to apply the skills they learned at the academy to improve their management of people, money and data on the job.
  • Academy participants were well prepared to respond to public health emergencies.

Funding
The Robert Wood Johnson Foundation (RWJF) provided $1,036,075 in funding from February 1998 to September 2003 to support the project.

 See Grant Detail & Contact Information
 Back to the Table of Contents


THE PROBLEM

A landmark 1988 study by the Institute of Medicine, The Future of Public Health, identified gaps in the performance of public health organizations and concluded that "greater emphasis should be placed on managerial and leadership skills." As this study and others by the CDC found, most public health professionals who enter management positions can demonstrate technical, programmatic or clinical expertise, but they often have little or no management training.

To ensure their success, state and local public health agencies need managers with the skills and competencies to confront multiple pressures of budgetary constraints and new responsibilities in the market-driven health care sector. The CDC estimated a target group of at least 12,000 public health professionals nationally in need of management training.

 Back to the Table of Contents


RWJF STRATEGY

This project relates to RWJF's objective of strengthening the leadership capacity and information infrastructure of the public health system. It builds on Turning Point: Collaborating for a New Century in Public Health, a national program launched in 1997 as a collaboration between the W.K. Kellogg Foundation and RWJF. The mission of Turning Point is to help states improve the performance of their public health functions through a state-level strategic assessment of public health's mission, its relationship to the private sector, organizational structure and governance, workforce capacity, accountability and sources of revenue. Turning Point operates through five national excellence collaboratives, one of which focuses on public health leadership development.

The Management Academy for Public Health project, which targets mid-level public health managers, also complements RWJF's State Health Leadership Initiative, which seeks to build the leadership capacity of upper-level state health officers as policy-makers, administrators and advocates for the health of the public.

 Back to the Table of Contents


THE PROJECT

From 1998 to 2003, the CDC Foundation, a nonprofit corporation established by Congress in 1992 to promote the mission of the CDC through public-private partnerships, administered a pilot project to design a management development program for public health professionals in four southeastern states (Georgia, North Carolina, South Carolina and Virginia).

In the development phase, the CDC Foundation contracted with two consultants (Cambridge, Mass.-based Elizabeth Nill, a management training expert, and McLean, Va.-based Nancy Rawding, a public health management expert) to conduct an extensive needs assessment (including focus groups with more than 250 public health officials), which concluded that the program should focus on mid-level managers, offering them a mix of strategic and technical skills training. The CDC Foundation sent a Request for Application based on these findings to accredited schools of public health and schools of business in the southeast. In 1999, after reviewing three applications (from the University of North Carolina at Chapel Hill (UNC), the University of South Carolina, and Virginia Commonwealth University), the Steering Committee and the CDC Foundation awarded a contract to implement the program to UNC's Kenan-Flagler School of Business and School of Public Health.

The overall goal of the education program was to improve the individual management competency of state and local public health professionals and the operational effectiveness and efficiency of their agencies. Key objectives were to:

  1. Design and evaluate a reasonably priced management development and training model that was sustainable over the long term.
  2. Provide management training to a pilot group of 600 managers in the four targeted states.
  3. Formulate a strategy to replicate components of the model nationwide.

Other Funding

The CDC Foundation received approximately $4 million in support of the project from four sponsors. These included RWJF and the W.K. Kellogg Foundation, which each awarded grants of $1,036,075, and the federal Health Resources and Services Administration and the CDC, which provided a total of $2,072,149 in support. The sponsors served as the project's Steering Committee. (See Appendix 1 for a list of Steering Committee members.) UNC contributed $374,384 in in-kind funding.

 Back to the Table of Contents


RESULTS

The project achieved results in the following areas:

  • UNC's Kenan-Flagler Business School and School of Public Health collaborated in designing the Management Academy for Public Health, a 10-month management development program providing the equivalent of 21 days of executive education. The curriculum focuses on the management of human resources, information systems and finances—described as "managing money, people and data." Training occurs in three phases:
    1. a summer five-day onsite session at UNC.
    2. a distributed learning phase that includes project work back home with coaching, individual self-development activities, online learning and a second onsite session.
    3. a final three-day onsite session and graduation.

    Participants attend in teams that consist of three to six managers from local and state agencies and often a community partner. To ensure transfer of management skills to the participants' agency, the teams create a public health business plan for a program in their community.
  • Over the four-year period, 639 managers enrolled in the academy and 593 (93 percent) graduated. Academy participants reflected the demographics of the public health workforce in the four-state pilot area. Seventy-five percent were female; 74 percent were white, 19 percent African-American and 2 percent Asian/Pacific Islander. One senior public health official served as the academy's representative in each of the four states, playing an important role in marketing the program and recruiting participants.

 Back to the Table of Contents


SIGNIFICANCE TO THE FIELD

The project team sees the Management Academy for Public Health as part of a broad national movement toward recognition of public health management as a specific field of academic inquiry and public health managers as a true professional group. Instead of grounding their management development and training program within a traditional public administration paradigm, the project team taught participants the concepts of "civic entrepreneurship." This approach breaks through traditional methods of thinking and managing in public health and encourages managers to form strategic alliances with other public and private organizations and to borrow private sector ideas for generating revenue.

 Back to the Table of Contents


EVALUATION

An internal evaluation, completed by staff at the Kenan-Flagler Business School and UNC's School of Public Health, focused on learning and reactions of academy participants as well as program structure and process before graduation. An external evaluation, conducted through a subcontract from the CDC Foundation with the Lewin Group, a health and human service consulting firm based in Falls Church, Va., focused primarily on the application of learning after graduation and its impact on participants' agencies and communities for the first three years of graduates, about 500 people.

The two evaluations asked complementary questions about the results of the academy; its on-the-job applicability; changes in the behavior, knowledge, confidence and beliefs of participants; their satisfaction with the program; and the effectiveness of the collaboration between the schools of business and public health. To answer these questions, the evaluators, both internal and external, used a combination of surveys, observations, interviews, reviews of business plans and site visits. The external evaluation also included questionnaires mailed to 452 academy graduates, 266 of whom responded (58.8 percent response rate) and work packets on revenue generation sent to 118 teams, 73 of whom responded (62 percent response rate).

The CDC Foundation subcontracted with the Georgia State University, Andrew Young School of Policy Studies, to create an executive summary weaving together findings of the two evaluations. An external evaluation committee provided input into the design and implementation of the evaluation. (See Appendix 2 for list of committee members.)

Evaluation Findings

Overall, the internal evaluation concluded that "after four years, the Management Academy for Public Health has established itself as an important, needed program that effectively prepares managers to transform the way their organizations manage people, money and data. Through the program, organizations are managing a difficult environment of constant change by planning and partnering in new ways." The Georgia State University executive summary highlighted the following key findings from the internal and external evaluations:

  • Trained at a total cost of $2 million, graduates of the first three years of the academy successfully applied their skills and competencies to garner over $6 million in revenue for their agencies. Twenty-eight of the teams who responded to the revenue-generation survey (38 percent of responding teams) were responsible for raising the estimated funds. Eight teams in Georgia produced over 40 percent of all enhanced revenues. Forty-five of the responding teams (62 percent) reported no enhanced revenue. Teams with revenue-generating initiatives raised an average of $216,000 each from a mix of grant and fee-based funding streams. Local agencies used funds to improve access to primary and preventive care for at-risk populations. The revenue-generating potential of the academy, an unintended but positive consequence of the pilot program, has since become a central strategy in UNC's marketing strategy for the academy.
  • After graduating from the academy, the majority of participants used their management skills to implement their business plans in their local health departments. Of 264 survey respondents, 54 percent were involved in implementing their plans at some level. Twenty-two percent of respondents (58 graduates) reported that they had fully implemented their business plans; another 32 percent were in the middle or beginning stages. Forty-six percent of respondents had postponed or abandoned their plans. Sample results from business plans in the four-state region included reducing child abuse and infant injury from misuse of car seats in Georgia, improving children's dental health in North Carolina and diabetes education in South Carolina.
  • The collaboration between the Kenan-Flagler Business School and School of Public Health of the University of North Carolina at Chapel Hill was a key factor in the success of the pilot project. Major driving forces in the success of the collaboration included a long-term strategic vision and unwavering commitment by the deans of the two schools, strong and committed program leadership, sufficient funding and resources and the presence of an ongoing quality-improvement process.
  • Overall, participants were able to apply the skills they learned at the academy to improve their management of people, money and data on the job. According to the internal evaluation, 94 percent of graduates indicated that they improved their on-the-job management practices, with increased skills in all 22 core areas taught in the academy. When transferring their new skills to the job, graduates were most successful in applying the seven skills related to managing people, which included communication, quality improvement, management of self and others, project management, negotiation and performance evaluation. Across the six data management skills, participants gave highest ratings to using data for decision-making, followed by analyzing and displaying data in effective presentations. Many participants found the six managing money skills less applicable to their job responsibilities. Although over 80 percent of respondents interacted more effectively with financial staff, for example, only half used their skills to prepare financial statements. Barriers to practical skills application included a nonsupportive organizational culture, lack of time and resources due to growing state budget deficits and lack of job applicability.
  • Academy participants were well prepared to respond to public health emergencies. Of 100 respondents to a December 2001 survey, 74 percent indicated that they were engaged in activities related to disasters and bioterrorism since September 11th, 2001. Ninety percent of those involved in such activities credited the academy with improving their performance in several critical skill areas that relate to disaster preparedness, especially partnership, people management, communication and strategy development skills.

Communications

Articles on the Management Academy for Public Health appeared in Public Health Reports and the Journal of Public Health Management and Practice. The project team made presentations on the program at the 2001 and 2003 annual meetings of the American Public Health Association, the National Association of County and City Health Officials and other national conferences. Articles, conference materials, evaluation reports and business plans created by academy participants are available on the academy Web site. See the Bibliography for details.

 Back to the Table of Contents


LESSONS LEARNED

The project generated the following lessons:

  1. Hiring a consultant with expertise in public health evaluation helped coordinate the internal and external evaluations. The consultant provided a bridge between the two groups of evaluators and ensured that the two studies complemented rather than overlapped each other. (Project Director/McGowan)
  2. Training programs that use work-based learning methods, such as the academy's business plan project, give students opportunities to apply their skills to the work environment. This model is more likely to produce lasting behavior changes than one that provides strong classroom training but then leaves students to their own devices to apply the content in their local agencies. (Project Director/Orton)

 Back to the Table of Contents


AFTER THE GRANT

Now in its fifth year, the Management Academy for Public Health has 99 enrolled scholars and is recruiting for a sixth year beginning in summer 2004. Subsidies from state and local governments and the CDC cover the $4,500 in tuition costs. For managers who cannot come to campus, UNC will offer training from the academy through an online certificate program beginning in 2004.

UNC's School of Public Health and Kenan-Flagler Business School received funding to conduct major training programs in other fields that are modeled on the academy. These include a grant from the W.K. Kellogg Foundation to recruit and train minority public health managers and a $3.6 million grant from the CDC to train teams of violence-prevention professionals. To help UNC formulate a long term strategy for sustaining the academy, RWJF approved a grant of $60,000 (ID# 045930) to the Colorado Foundation for Public Health to explore the feasibility of advancing the work of the academy beyond the four targeted states.

 Back to the Table of Contents


GRANT DETAILS & CONTACT INFORMATION

Project

Pilot Educational Network for the Advancement of Public Health Program Management

Grantee

National Foundation for the Centers for Disease Control and Prevention (Atlanta,  GA)

  • Amount: $ 1,036,075
    Dates: February 1998 to September 2003
    ID#:  033489

Contact

Linda Kay McGowan
(404) 653-0790
lkm3@cdc.gov
Stephen Orton, Ph.D.
(919) 966-8125
stephen_orton@unc.edu

Web Site

http://www.maph.unc.edu

 Back to the Table of Contents


APPENDICES


Appendix 1

(Current as of the time of the grant; provided by the grantee organization; not verified by RWJF.)

Management Academy for Public Health Steering Committee

Edward L. Baker, M.D., M.P.H.
Director, North Carolina Institute for Public Health
Research Professor, Department of Health Policy and Administration
University of North Carolina
School of Public Health
Chapel Hill, N.C.

Claude Earl Fox, M.D., M.P.H.
Administrator
Health Resources and Services Administration
Rockville, Md.

Susan Hassmiller, Ph.D., R.N.
Senior Program Officer
The Robert W. Johnson Foundation
Princeton, N.J.

Barbara Sabol
Program Director
W. K. Kellogg Foundation
Battle Creek, Mich.

Sam Shekar, M.D., M.P.H. (final stage of project)
Associate Administrator for Health Professions
Health Resources and Services Administration
Rockville, Md.


Appendix 2

(Current as of the time of the grant; provided by the grantee organization; not verified by RWJF.)

Management Academy for Public Health External Evaluation Committee

Judith Ottoson
Associate Professor
Department of Urban Life
Georgia State University
Atlanta, Ga.

Linda Kay McGowan
Vice President of Programs
CDC Foundation
Atlanta, Ga.

Richard House, M.P.H., Ed.D.
CDC Foundation Fellow
CDC Foundation
Atlanta, Ga.

Georgina Howard, R.N., M.S.
Director
Personal Health Services
DeKalb County Board of Health
Decatur, Ga.

Jeffrey Lake, M.S.
Associate Commissioner
Office of Community Health Services
Virginia Department of Health
Richmond, Va.

Ben Rosen
Hanes Professor of Management
University of North Carolina at Chapel Hill
Kenan-Flagler Business School
Chapel Hill, N.C.

Karl E. Umble, Ph.D., M.P.H.
North Carolina Institute for Public Health
School of Public Health
University of North Carolina
Chapel Hill, N.C.

Jeff Dunlap
Acting Director for the Center of Public Health
Health Resources and Services Administration
Rockville, Md.

Colleen Hirschkorn
Senior Vice President
Lewin Group
Falls Church, Va.

Terry West
Senior Manager
Lewin Group
Falls Church, Va.

Laura C. Leviton, Ph.D.
Senior Program Officer
The Robert Wood Johnson Foundation
Princeton, N.J.

Paul Halverson, Dr.P.H.
Director, Public Health Systems Development and Research
Public Health Practice Program Office
Centers for Disease Control and Prevention
Atlanta, Ga.

 Back to the Table of Contents


BIBLIOGRAPHY

(Current as of date of this report; as provided by grantee organization; not verified by RWJF; items not available from RWJF.)

Articles

Orton S, Umble K, Davis M and Port J. "Disasters and Bioterrorism: Does Management Training Develop Readiness?" Public Health Reports, 117(6): 596–598, 2002. Also appears online.

Porter J, Johnson J, Upshaw VM, Orton S, Deal KM, and Umble K. "The Management Academy for Public Health: A New Paradigm for Public Health Management Development." Journal of Public Health Management and Practice, 8(2): 66–78, 2002. Abstract available online.

Setliff R, Porter JE, Malison M, Frederick S and Balderson TR. "Strengthening the Public Health Workforce: Three CDC Programs That Prepare Managers and Leaders for the Challenges of the 21st Century." Journal of Public Health Management and Practice, 9(2): 91–102, 2003. Abstract available online.

Reports

Evaluation Update—Year Two Results. Chapel Hill, NC: Management Academy for Public Health, University of North Carolina at Chapel Hill, April 2001.

Evaluation Update—Individual and Team Development. Chapel Hill, NC: Management Academy for Public Health, University of North Carolina at Chapel Hill, November 2001.

Evaluation Update-Thinking Business, Thinking Teams. Chapel Hill, NC: Management Academy for Public Health, University of North Carolina at Chapel Hill, February 2003.

Management Academy for Public Health: Final Program Evaluation. Falls Church, VA: Lewin Group, June 20, 2003.

Ottoson JM. Executive Summary of the Internal and External Evaluation Reports of the Management Academy for Public Health. Atlanta: Andrew Young School of Policy Studies, Georgia State University, September 2003.

Thielen L. The Sustainability of the Management Academy for Public Health. Colorado Foundation for Public Health and Environment, July 2003.

Umble KE and Orton SN. Management Academy for Public Health: Internal Evaluation Final Report. Chapel Hill, NC: North Carolina Institute for Public Health, School of Public Health, University of North Carolina at Chapel Hill, August 2003.

World Wide Web Sites

www.maph.unc.edu. The Web site of the Management Academy for Public Health make available articles, conference materials, overhead slides, posters, evaluation reports and business plans created by MAPH participants. Chapel Hill, NC: University of North Carolina at Chapel Hill.

Presentations and Testimony

Karl Umble, Vaughn Upshaw, Stephen Orton and Kelly Matthews, "Using the Post-Then Method to Assess Learner Change," at the AAHE Assessment Conference, June 15, 2000, Charlotte, NC. Text of the presentation is available online.

Stephen Orton and Janet L. Place, "Lessons Learned in Meeting the Training Needs of the Diverse Public Health Workforce," at the 2001 Annual Meeting of the American Public Health Association, October 21–25, 2001, Atlanta. Slide presentation available from the author.

Stephen Orton, "Tools for the Business of Collaboration," at the Health Communities Institute, June 5, 2003, Asheville, NC. Slide presentation available from the author.

Stephen Orton, "Business Planning in Public Health," at the annual conference of the National Association of County and City Health Officials, September 12, 2003, Phoenix. Presentation overheads available online.

Stephen Orton, Karl Umble, Janet Porter, James Johnson and Thomas Ricketts, "Civic Entrepreneurship in Community Health Planning: An Idea Whose Time has Come," at the 131st Annual Meeting of the American Public Health Association, November 17, 2003, San Francisco. Poster available at online.

 Back to the Table of Contents


Report prepared by: Jayme Hannay
Reviewed by: Robert Crum
Reviewed by: Molly McKaughan
Program Officer: Susan Hassmiller

Most Requested