July 2010

Grant Results

SUMMARY

In 2006, a research team based at the University of Arkansas and the University of Kentucky replicated a 1998 survey of large public health systems in order to develop an evidence-based typology of the structure and dynamics of local public health delivery systems. The purpose was to provide researchers and policy-makers with a tool to help identify better ways of organizing and delivering public health services at the local level.

To disseminate its findings, the team published an issue brief and journal articles and organized the first Keeneland Conference on public health services and systems research and its applications.

Key Findings

  • Local public health systems can be classified into seven distinct categories (termed clusters), based on their organizational structure. The classification is based on three factors:
    • The scope of services provided
    • The range of governmental and nongovernmental organizations providing the services
    • The extent to which the governmental public health agency bears the brunt of the delivery effort versus spreading the responsibility across the contributing organizations
  • Local public health systems surveyed for the project frequently migrated across the classifications, trending toward a broader scope of services and engagement with a wider range of organizations.

Funding

The Robert Wood Johnson Foundation (RWJF) funded the project with two grants totaling $289,640:

  • A $237,584 grant in August 2005 supported development of the typology (ID# 053229)
  • A $52,056 grant in October 2006 funded dissemination activities (ID# 059229).

 See Grant Detail & Contact Information
 Back to the Table of Contents


Context

Studies show that public health services vary widely across the nation in availability and effectiveness—and in many communities are far from optimal. The threats of bioterrorism, pandemic disease and other emerging health dangers have focused increased attention on this disparity in public health system performance.

A 2002 report by the Institute of Medicine (The Future of the Public's Health in the 21st Century) noted that little is known about the structure and operation of public health systems and how that affects the availability and quality of public health services.

RWJF's Interest in the Area

Ensuring that all Americans have access to quality health services is a major objective of RWJF. Driving fundamental improvements in the performance and impact of public health agencies is a key strategy to reach that goal. Among its concurrent initiatives in this area were the following:

  • The National Association of County and City Health Officials was developing an operational definition of a functional local health department, with associated performance measures. See Grant Results.
  • A collaborative effort by that organization and the Association of State and Territorial Health Officers was laying the groundwork for a voluntary national accreditation program for state, local, territorial and tribal public health departments. See Grant Results.

RWJF's Public Health team believed that devising a typology of public health systems—classifying them according to their structure and organization—would be a complement to these activities. Typologies in other areas—managed care organizations, hospital networks—have served a useful role in policy and administrative decisions. The team believed that a similar typology for public health systems would facilitate assessment of the strengths and weaknesses of those systems and the impact of their structure and operations on public health.

 Back to the Table of Contents


The Project

Under the first grant of this project (ID# 053229), a research team replicated a 1998 survey of large public health systems in order to develop an evidence-based typology of the structure and dynamics of local public health delivery systems. Researchers also sought to examine:

  • How public health systems change in structure over time
  • How system performance varies across different kinds of organizational structures

The co-principal investigator was Glen P. Mays, Ph.D., M.P.H., professor in the Fay W. Boozman College of Public Health at the University of Arkansas for Medical Sciences.

F. Douglas Scutchfield, M.D., professor in the University of Kentucky College of Public Health, was co-investigator.

The 1998 survey was mailed to directors of the 497 local health agencies serving jurisdictions with a population of at least 100,000. The researchers surveyed the same agencies in 2006. Response rates were 78 percent in 1998 and 70 percent in 2006. The survey instrument asked about:

  • The scope of local public health services—specifically, the availability of 20 activities deemed important to protecting and improving public health at the community level
  • The range of organizations (public and private) involved in performing these activities
  • The level of effort contributed by the governmental public health agency relative to other organizations in the community

The researchers' analysis relied on the responses to surveys as well as census and other secondary data. They also conducted interviews with local public health officials to determine how different types of systems operate and evolve.

Communications

RWJF provided a second grant (ID# 059229) to support expanded communication efforts, including a national conference. Communications activities supported by both grants included:

  • A 2008 issue brief, Finding Order in Complexity: A Typology of Local Public Health Delivery Systems, available online.
  • Articles about the work in peer-reviewed journals, including: See the Bibliography for details and links to some of the articles.
  • Presentations at professional conferences, including annual meetings of the American Public Health Association and AcademyHealth.
  • The first Keeneland Conference, devoted to public health services and systems research and its applications. The conference, now an annual event, took place in Lexington, Ky., April 9–11, 2008, and drew 135 public health researchers, practitioners and policy-makers.
  • Assisting scholars at the universities of Washington, North Carolina, Iowa and Tennessee and other institutions in applying the typology to their own research.

See the Bibliography for details of many of these communications activities.

 Back to the Table of Contents


Findings

The researchers reported the following findings in an article, "Understanding the Organization of Public Health Delivery Systems: An Empirical Typology," published in the March 2010 issue of the Milbank Quarterly and an issue brief, Finding Order in Complexity: A Typology of Local Public Health Delivery Systems:

  • Local public health systems can be classified into seven distinct categories (termed clusters), based on their organizational structure. Although public health systems vary widely in structure, this typology "contradicts the often-cited anecdote that 'if you've seen one health department, you've seen one health department,'" the researchers wrote in the Milbank Quarterly article. The typology is based on three factors:
    • The scope of services provided. The team termed this measure differentiation. A highly differentiated system is one providing a broad scope of services.
    • The range of organizations providing the services—termed integration. Highly integrated systems engage a wide range of contributors.
    • The extent to which the governmental public health agency bears the brunt of the delivery effort versus spreading the responsibility across the contributing organizations—termed concentration. A concentrated system relies heavily on the governmental agency.
    The following table summarizes the typology, groups the clusters into three tiers based on the scope of services provided and shows the percentages of surveyed systems falling into each cluster.

    Summary of the Local Public Health System Typology
    Type of System & PrevalenceDescription
    Tier I: Comprehensive Systems
    1. Concentrated Comprehensive
    • 1998: 12.5%
    • 2006: 21.4%
    • Broad scope of activities are performed
    • Wide range of organizations participate in activities
    • Local public health agency shoulders much of the effort in performing activities
    2. Distributed Comprehensive
    • 1998: 5.1%
    • 2006: 3.9%
    • Broad scope of activities are performed
    • Wide range of organizations participate in activities
    • Effort in performing activities is distributed across participating organizations
    3. Independent Comprehensive
    • 1998: 6.6%
    • 2006: 11.6%
    • Broad scope of activities are performed
    • Narrow range of organizations participate in activities
    • Local public health agency shoulders much of the effort in performing activities
    Tier II. Conventional Systems
    4. Concentrated Conventional
    (Transitory System)
    • 1998: 3.4%
    • 2006: 3.0%
    • Moderate scope of activities are performed
    • Moderate range of organizations participate in activities
    • Local public health agency shoulders much of the effort
    • Highly transitory system
    5. Distributed Conventional
    (Modal System)
    • 1998: 46.7%
    • 2006: 30.9%
    • Moderate scope of activities are performed
    • Moderate range of organizations participate in activities
    • Effort in performing activities is distributed across participating organizations
    Tier III. Limited Systems
    6. Concentrated Limited
    • 1998: 12.3%
    • 2006: 18.0%
    • Narrow scope of activities are performed
    • Limited range of organizations participate in activities
    • Local public health agency shoulders much of the effort in performing activities
    7. Distributed Limited
    • 1998: 13.4%
    • 2006: 11.2%
    • Narrow scope of activities are performed
    • Moderate range of organizations participate in activities
    • Effort in performing activities is distributed across participating organizations
  • Many local public health systems moved from one cluster to another between 1998 and 2006, with a trend toward offering a broader scope of services and engaging a wider range of organizations. In particular:
    • On average, systems performed 64 percent of studied activities in 1998 and 70 percent in 2006.
    • The breadth of organizations contributing to public health activities increased moderately between 1998 and 2006. Community health centers, federal agencies and educational institutions showed the largest increase in contributions to at least one activity.
    It was nonetheless the case that some systems reduced their range of activities and number of partners.
  • Subsequent research using the typology, presented at the 2008 AcademyHealth Annual Research Meeting and included in a forthcoming article, found that local public health officials rated highly differentiated and integrated public health systems significantly higher than their counterparts on measures of the perceived effectiveness of public health activities in addressing community needs. Similar analyses found significantly larger reductions in rates of preventable mortality among systems that became more integrated and differentiated over the 1998 to 2006 period of study, compared with systems that remained static or became less integrated and differentiated.

Limitations

The researchers noted the following limitations of the study in the Milbank Quarterly article:

  • The typology was derived from a study of the nation's largest public health systems—those serving at least 100,000 people—and the extent to which the categorizations apply to smaller systems is unknown.
  • The typology does not include structural characteristics related to financing, workforce, governance and other factors likely to be important to understanding the organization and operation of public health systems.
  • The team did not attempt to identify an optimal organizational configuration for public health systems or the circumstances in which a given configuration performs best. However, the typology provides a "starting point for conducting the comparative research" to answer questions of this kind.

 Back to the Table of Contents


Lessons Learned

The project team reported the following lessons to RWJF:

  1. When studying a complex phenomenon such as public health delivery systems, use a multipronged, mixed-method research design that draws on both quantitative and qualitative approaches. Survey data allowed the team to identify system types. However, understanding the similarities and differences across these types and their evolution over time required qualitative information that came from interviews with public health officials. (Co-Principal Investigator/Mays)
  2. Realize the value of disseminating findings through multiple venues and channels. The issue brief allowed the team to get its findings to the field faster than the traditional route of peer-reviewed journal articles, whereas articles were important in securing credibility for the findings and reaching broad segments of the targeted audience. (Co-Principal Investigator/Mays)
  3. Use multiple presentations to demonstrate different applications of research findings. In presentations at successive American Public Health Association and AcademyHealth meetings, team members focused on varying uses—and limitations—of the typology. Making the content unique to each appearance generated greater interest in the typology than was likely had the material been more static. (Co-Principal Investigator/Mays)

 Back to the Table of Contents


Afterward

The project team reported that the typology and the data gathered in this project were employed in a variety of contexts, including:

  • RWJF's national program Changes in Health Care Financing and Organization provided funding to use the data sources from this project to study changes in local public health spending.
  • The state of Arkansas funded use of the typology to examine statewide variation in local public health systems' structure and performance. The work was in conjunction with the state's adoption of the National Public Health Performance Standards Program (NPHPSP) of the federal Centers for Disease Control and Prevention (CDC).
  • Researchers at a number of other universities, including the University of Kentucky, the University of Tennessee and the University of Georgia, are using the typology for ongoing studies and dissertations related to public health system performance. Kentucky received CDC funding to use the typology to analyze variations in system performance based on NPHPSP data and most recently has been developing a project to examine how public health laws influence system typologies.

Co-Principal Investigator Mays directs the Robert Wood Johnson Foundation Practice-Based Research Network in Public Health—an RWJF-funded program to support the development of research networks for studying public health practice. Several networks are using the typology as part of comparative research studies examining the organization and delivery of public health services, according to Mays.

RWJF has continued funding the Keeneland Conferences, hosted by the Center for Public Health Systems & Services Research at the University of Kentucky.

 Back to the Table of Contents


GRANT DETAILS & CONTACT INFORMATION

Project

National study aimed at developing a typology of the structure and dynamics of the public health delivery system

Grantee

University of Arkansas for Medical Sciences (Little Rock,  AR)

  • National study aimed at developing a typology of the structure and dynamics of the public health delivery system
    Amount: $ 237,584
    Dates: August 2005 to January 2008
    ID#:  053229

  • National meeting aimed at disseminating an evidence-based typology of the structure and dynamics of the public health delivery system
    Amount: $ 52,056
    Dates: October 2006 to April 2008
    ID#:  059229

Contact

Glen P. Mays, Ph.D., M.P.H.
(501) 526-6633
gpmays@uams.edu

 Back to the Table of Contents


BIBLIOGRAPHY

(Current as of date of this report; as provided by grantee organization; not verified by RWJF; items not available from RWJF.)

Articles

Bhandari MW, Scutchfield FD, Charnigo R, Riddell MC and Mays GP. "New Data, Same Story? Revising Studies on the Relationship of Local Public Health Systems Characteristics to Public Health Performance." Journal of Public Health Management and Practice, 16(2): 110–117, 2010. Abstract available on RWJF's Web site.

Mays GP, Scutchfield FD, Bhandari MW and Smith SA. "Understanding the Organization of Public Health Delivery Systems: An Empirical Typology." Milbank Quarterly, 88(1): 81–111, March 2010. Abstract available on RWJF's Web site, with link to full article.

Mays GP, Scutchfield FD, Bhandari MW and Smith SA. "Finding Order in Complexity: an Empirical Typology of Local Public Health Delivery Systems." Unpublished (submitted).

Mays GP and Smith SA. "Geographic Variation in Public Health Spending: Correlates and Consequences." Health Services Research, 44(5.2): 1796–1817, August 2009. Part of a special issue available on RWJF's Web site.

Mays GP, Smith SA, Ingram R, Racster L, Lamberth C and Lovley E. "Public Health Delivery Systems: Evidence, Uncertainty and Emerging Research Needs." American Journal of Preventive Medicine, 36(3): 256–265, March 2009. Abstract available online.

Scutchfield FD, Marks JS, Perez DJ and Mays GP. "Public Health Services and Systems Research." American Journal of Preventive Medicine, 33(2): 169–171, 2007.

Scutchfield FD, Mays GP and Lurie N. "Applying Health Services Research to Public Health Practice: An Emerging Priority." (Editorial.) Health Services Research, 44(5.2): 1775–1787, August 2009. Part of a special issue available on RWJF's Web site.

Scutchfield FD, Mays GP and Perez D (eds). "Special Issue: Advances in Public Health Services and Systems Research." Health Services Research. In press.

Reports

Mays GP, Scutchfield FD, Bhandari MW and Smith SA. "Finding Order in Complexity: A Method for Classifying and Comparing Local Public Health Delivery Systems." Issue Brief: Findings from Public Health Systems and Services Research. Little Rock, AR: University of Arkansas for Medical Sciences; January 2008.

Mays GP and Smith SA. National Longitudinal Survey of Local Public Health Systems: Customized Report of Results. Little Rock, AR: University of Arkansas for Medical Sciences; January 2007.

Mays GP. Understanding the Dimensions of Public Health Delivery Systems: Theory, Evidence, and Unanswered Questions. Paper prepared for the AcademyHealth Invitational Conference on Public Health Systems Research. Washington: AcademyHealth; June 2007.

Presentations and Testimony

Mays GP. "Engaging Public Health Agencies the Research Enterprise: Models for Practice-based Research from a New National Initiative" at the 2008 Annual Meeting of the American Public Health Association, San Diego, CA, October 2008. Proceedings of the meeting available online.

Mays GP. "Organizational Borders and Local Public Health Delivery Systems: Using an Empirical Typology to Assess Systems Change and Health Outcomes" at the 2008 Annual Meeting of the American Public Health Association, San Diego, CA, October 2008. Proceedings of the meeting available online.

Mays GP. "The Dynamics of Public Health Infrastructure: Using a Delivery System Typology to Assess Structural Change and Health Outcomes" at the 2008 AcademyHealth Annual Research Meeting, Washington, June 2008.

Mays GP. "The Dynamics of Public Health Infrastructure: Using a Delivery System Typology to Assess Structural Change and Health Outcomes" at the University of Tennessee Public Health Colloquium. Written invitation from sponsor Dr. Paul Erwin, Knoxville, TN, April 17, 2008.

Mays GP. "Structure and Dynamics of Public Health Delivery Systems: Pathways for Health Improvement" at the Arkansas Department of Health Public Health Grand Rounds. Written invitation from department director Paul Halverson. Little Rock, AR, February 14, 2008.

Scutchfield FD, Mays GP, Bhandari MW and Smith SA. "Developing a Typology of Public Health Systems and Dynamics: Qualitative Analysis" at the 2007 Annual Meeting of the American Public Health Association, Washington, November 7, 2007. Proceedings of the meeting available online.

Mays GP. "Moving Public Health Systems Research from Infancy to Adolescence: Research Needs, Stakeholder Priorities, and a Joint Agenda" at the 2007 Annual Meeting of the American Public Health Association, Medical Care Section, Washington, November 5, 2007. Proceedings of the meeting available online.

Mays GP. "Public Health Systems Research: Creating an Agenda to Improve Policy and Practice" at the Fall 2007 Regional Meeting of the Association for State and Territorial Health Officials, Little Rock, AR, October, 2007.

Mays GP. "Understanding the Dimensions of Public Health Delivery Systems: Theory, Evidence, and Remaining Questions" at the AcademyHealth Invitational Conference on Public Health Systems Research, Washington, June 2007.

Mays GP and Smith SA. "Complex and Adaptive Systems: Changes in the Organizations Contributing to Local Public Activities" at the 2007 AcademyHealth Annual Research Meeting, Orlando, FL, June 2007.

Mays GP. "Public Health Systems Research: Evolution of the Field" at the U.S. Department of Health and Human Services Public Health Expert Panel Meeting, Washington, February 8, 2007. Written invitation from the panel chair Garth Graham.

Mays GP. "Structure and Dynamics of Public Health Delivery Systems" at 2006 American Public Health Association Annual Meeting, Boston, November 7, 2006. Proceedings of the meeting available online.

Mays GP. "Assessing Alternatives for Organizing and Delivering Public Health Services." Testimony given to the Joint Public Health Committee of the Arkansas 85th General Assembly, July 10, 2006. Written invitation from the committee chair, Sen. Jack Critcher. Proceedings available online.

Mays GP. "Developing an Evidence-based Typology of Public Health System Structure and Dynamics" at the 2006 AcademyHealth Annual Research Meeting, Public Health Systems Interest Group, Seattle, June 25, 2006.

Mays GP. "Disparities in Public Health Resources in the Delta: Implications for Regionalization" at the 2006 AcademyHealth National Health Policy Conference, Affiliate Meeting on Public Health Systems Research, Washington, February 12, 2006.

Survey Instruments

Mays GP. "National Longitudinal Survey of Local Public Health Systems." University of Arkansas for Medical Sciences, fielded May 28–September 30, 2006.

Computer-Readable Data Tapes and Documentation

"National Longitudinal Survey of Local Public Health Systems," Boozman College of Public Health, University of Arkansas for Medical Sciences. May–October 1998 and May–October 2006. Prepared for submission to ICPSR.

Sponsored Conferences

"Keeneland Conference on Public Health Services and Systems Research," April 9–11, 2008, Lexington, KY. Co-sponsored with the University of Kentucky College of Public Health. Attendance by 135 registrants. Examples of organizations represented include U.S. Centers for Disease Control and Prevention, U.S. Agency for Healthcare Research and Quality, National Association of County and City Health Officials. Proceedings of the meeting available online.

 Back to the Table of Contents


Report prepared by: Michael H. Brown
Reviewed by: Robert Narus
Reviewed by: Molly McKaughan
Program Officer: Debra J. Perez

Most Requested