July 2005

Grant Results

SUMMARY

The Task Force for Child Survival and Development in Decatur, Ga., worked for four years (2000–2004) with federal, state and local public health agencies across the nation to foster the development of computerized child health information systems that integrate data from multiple sources.

In response to low immunization rates among preschool children, the Robert Wood Johnson Foundation (RWJF) in 1991 initiated a national program called All Kids Count: Establishing Immunization Monitoring and Follow-up Systems to create immunization registries (for more information see Grant Results).

An immunization registry is a computerized information system that collects child vaccination records from multiple health care providers in a geographic area.

In July 2000, the RWJF Board of Trustees funded an All Kids Count technical resource center.

The project staff formed a collaborative of health organizations working to integrate child health information systems. The goal of the collaborative, called Connections, was to provide a forum for members to learn from one another and to document best practices.

Members were eligible for mini-grants to support their local integration initiatives. In addition to Connections, which continued with federal funding after the project ended, the staff convened an invitational conference and produced various publications to encourage the development of integrated child health information systems.

Key Results

  • The technical resource center strengthened the public health sector's involvement in and commitment to the development of integrated child health information systems.
  • The concept of integrating immunization registries with other child health information systems gained increased support from federal agencies — including the Health Resources and Services Administration, the Centers for Disease Control and Prevention and the Centers for Medicare & Medicaid Services.
  • The technical resource center established a community of practice dedicated to improving public health information networking.
  • The technical assistance funds provided to Connections members stimulated solutions to networking issues that were shared with the health community at large.
  • The technical resource center helped move immunization registries and integrated child health information systems beyond the concept stage.

Funding
RWJF supported the All Kids Count project with a $4,999,270 grant between August 2000 and July 2004.

 See Grant Detail & Contact Information
 Back to the Table of Contents


THE PROBLEM

Integrated child health information systems consolidate data on multiple health care services that children receive from various sources in a community. These computerized systems help public health agencies evaluate the health status of children and identify gaps where additional services are needed. (For a more comprehensive explanation of integrated systems and their potential benefits to health care providers and families as well as public health agencies, see Appendix 1.)

There are barriers to implementing integrated child health information systems, however. One major challenge is that, historically, state and local health agencies have developed separate information systems for different diseases and health programs. The result has been a proliferation of what are known as information "silos" — unconnected, incompatible single-focus data systems that are unable to exchange information with each other.

While technology to link separate systems exists, the coordination necessary to set standards for systems architecture and data collection policies has been lacking. Setting privacy guidelines to protect the confidentiality of the information and identifying funding mechanisms to sustain integrated systems also have proved difficult.

The public health community's interest in integrated information is part of an effort by the entire health sector to move more decisively into the computer age. Health care systems, hospitals and individual providers increasingly are investing in electronic medical record systems as a means of improving the safety, quality and efficiency of care delivery. In the late 1990s, the National Committee on Vital and Health Statistics, a federal advisory committee to the U.S. Department of Health and Human Services (DHHS), initiated work on the National Health Information Infrastructure — a concept for pulling health information networks serving both clinical medicine and public health into one coordinated framework.

 Back to the Table of Contents


RWJF STRATEGY

In response to low immunization rates among preschool children, RWJF in 1991 initiated a national program called All Kids Count: Establishing Immunization Monitoring and Follow-up Systems, to create immunization registries. An immunization registry is a computerized information system that collects child vaccination records from multiple health care providers in a geographic area.

For public health authorities, an immunization registry is a useful tool to assess vaccination coverage rates and identify pockets of underimmunization where children could be at risk for vaccine-preventable diseases. A registry also allows health care providers to identify the immunization needs of specific patients and send out notices when shots are due — an especially valuable tool to track children whose families are medically mobile. An estimated 25 percent of American children under age three receive their immunizations from more than one provider, in part because their families move about.

RWJF established a National Program Office to manage the work of All Kids Count at the Task Force for Child Survival and Development, a nonprofit public health organization based in Decatur, Ga., an Atlanta suburb. Phase I of the program (1991–1997) supported initial development work on 24 state and local registries. Phase II (1997–2001) provided implementation grants to 16 registry projects. Altogether, RWJF invested more than $20 million in the effort. For a comprehensive description of both phases, see the All Kids Count Grant Results.

As Phase II neared its end, a number of the All Kids Count grantees were interested in connecting immunization registries to other child health information systems. Health care organizations not directly involved in the program also sought leadership from the national program staff.

As a result, in June 2000, the task force proposed a Phase III of All Kids Count to stimulate development of integrated child health information systems to facilitate the assessment for and prompt provision of requisite services to ensure an optimal healthy start for all children and improve the health of children.

To achieve this outcome, integrated child health information systems must share information and knowledge appropriately and efficiently so that it is readily available to patients, families, health care and public health professionals and public agencies. This challenge is especially important for children who are born with heritable and congenital conditions identified through newborn dried blood spot screening and universal newborn hearing screening.

As a starting point, the principal focus was on four programmatic areas: newborn dried blood spot screening (a blood test for certain metabolic and other inherited disorders); early hearing detection and intervention; immunizations and vital registration; as well as information from Medicaid and the Special Supplemental Nutrition Program for Women, Infants and Children (WIC).

The proposal called for the All Kids Count staff to work with public agencies and private organizations to develop a "shared vision" of integrated preventive child health information systems. The vision was to include:

  1. Standards to govern the sharing of data between systems.
  2. Confidentiality guidelines to protect people's privacy.
  3. An analysis of the costs and benefits of integrated systems and funding options.
  4. Strategies for increasing private provider participation in public health information systems.

RWJF program staff considered All Kids Count Phases I and II to have been successful in demonstrating the feasibility and effectiveness of immunization registries and believed the proposed new phase, to establish a resource and technical assistance center, would build on that earlier work and on the credibility that the program office had established in the health information field. In July 2000, the Board of Trustees approved a $5-million grant (ID# 039374) to fund the All Kids Count technical resource center for three years, later extended to four years.

 Back to the Table of Contents


THE PROJECT

Laying the Groundwork
In addition to its broadened focus, the technical resource center differed in two key respects from the earlier All Kids Count program. First, unlike the first two phases, it did not entail separate RWJF funding for pilot projects. Instead, the $5-million grant included $1.3 million to be disbursed by the task force staff as technical assistance mini-grants to organizations working on integrated systems. (Of the remaining $3.7 million, $2.4 million was to support the project staff and $1.3 million was to cover expenses for communications, travel, meetings and consultants.)

Second, without pilot projects to select and oversee, the approximately 12-member All Kids Count staff concentrated its efforts more heavily on providing technical assistance to and building partnerships with government agencies and nongovernment health organizations. Reflecting that shift, the staff traded its designation as a National Program Office for the new classification of Technical Resource Center.

While much of the Phase II staff remained unchanged, David A. Ross, Sc.D., an expert in health information systems, took over as All Kids Count director. (The program had been directed by William H. Foege, M.D., M.P.H., from December 1991 through July 1997 and by William C. Watson, Jr., M.P.A., from August 1997 through July 2001.) Ross, a retired U.S. Public Health Service officer, had been assistant director for information and communications services at the federal Centers for Disease Control and Prevention (CDC) and more recently, an executive with the Cerner Corp., a supplier of health care information technology.

The staff spent the first six months researching child health information systems and meeting with government agencies and private organizations involved in child health programs to discuss integration efforts. These included the following:

  • American Academy of Pediatrics
  • Association for State and Territorial Heath Officials
  • CDC
  • Health Care Financing Administration, now Centers for Medicare & Medicaid Services (CMS)
  • Maternal and Child Health Bureau of the Health Resources and Services Administration (HRSA/MCHB)
  • National Association of County and City Health Officials
  • National Center for Health Statistics

The staff hired a consultant to research medical software systems used in preventive child health programs and convened a working group. This group of information system experts and former All Kids Count grantees was to consider information system models and the costs and benefits of integration.

The technical resource center staff defined technical assistance broadly to cover a wide range of activities, including educating health policy leaders on integration issues, documenting and disseminating best practices in the integration of child health information systems and helping organizations during their planning and implementation phases.

For example, the project staff contracted with the American Immunization Registry Association, a membership organization that grew out of All Kids Count Phase II, to conduct workshops on information integration. It also sponsored a meeting on linkages between immunization registries and the Vaccine Adverse Event Reporting System, a cooperative safety program of the CDC and the U.S. Food and Drug Administration.

To gain greater knowledge of the field, the staff surveyed 18 public health departments involved in integration efforts around the nation about their information systems. Staff also conducted telephone interviews with parents of children with special health care problems to learn their needs for and perspective on child health information.

Connections: The Major Activity
To stimulate integrated child health information systems, provide technical assistance and build partnerships, the project staff organized a forum or collaborative — named Connections — open to public health agencies and others interested in working cooperatively to advance the field of information integration. The collaborative was not part of the grant proposal, but it quickly evolved into the principal technical resource center activity and focus.

The staff saw Connections as a means of bringing states and communities together to share their experiences and knowledge on a peer-to-peer basis and to identify best practices in data integration. As Ross explains, Connections provided a trusted, learning environment that encouraged stakeholders to share their failures as well as their successes. He characterized Connections as a community of practice, a term for an informal group whose participants share a common pursuit.

Connections attracted 11 members. Some had been All Kids Count Phase II grantees, and some had federal funding from HRSA/MCHB to work on information integration projects. (See HRSA: A Key Partnership.) All were involved in linking at least two child health information systems.

One member was CalOptima, a health system that administers state Medicaid benefits in Orange County, Calif. The other 10 were public health agencies:

  • Iowa Department of Public Health
  • Kansas Integrated Public Health System
  • Maine Department of Health and Human Services
  • Michigan Department of Community Health
  • Missouri Department of Health and Senior Services
  • New York City Department of Health and Mental Hygiene
  • Oregon Department of Human Services
  • Rhode Island Department of Health
  • Santa Clara County (California) Health Department
  • Utah Department of Health

It was to members of Connections — and only members were eligible — that the project staff offered technical assistance mini-grants. In May 2001, the project staff issued a Call for Proposals soliciting projects aimed at developing some aspect of an integrated child health information system. In addition to testing technology to link disparate systems, projects could focus on less technical tasks, such as:

  • Developing relationships between the public health and private health care sectors to bring together programs and systems.
  • Making a business case for the integration of child health information systems.
  • Analyzing privacy and security issues.
  • Planning an evaluation of the impact of an integrated system.

The staff conducted two competitive funding rounds and awarded a total of 16 grants (the term grants was used; but legally they were contracts). Fourteen grants of about $50,000 each went to single members of Connections, and two grants totaling $104,000 supported projects involving multiple Connections members. Five members of Connections received grants in both funding rounds, while two members received no grant funding.

The funded Connections projects differed widely. For example, one project examined different software products that clean data to facilitate merger of electronic records. Another analyzed the business case for an integrated information system. A third created a marketing plan to promote the use of an integrated information by health care providers and families. (See Appendix 2 for a description of all 16 projects.)

Connections held its initial meeting in June 2001 in Atlanta and thereafter met at five different members' sites. Each site meeting lasted two-and-a-half days, split evenly between learning about the project of the host organization and considering more general integration issues.

Representatives of federal public health agencies participated in Connections as ad hoc members, and consultants to the member organizations and other invitees also attended meetings. For example, one session featured a panel of health care providers. Between meetings, the staff and Connections members communicated through conference calls, a listserv and an interactive Web site. The staff used various communications mechanisms to share the knowledge developed by Connections with stakeholders and other interested parties. (See Communications for an account of these dissemination efforts.)

Conference Planning: Another Key Activity
To develop a shared vision of information integration, the grant proposal called for the project to produce a white paper endorsed by key stakeholder groups in the public and private sector. However, in November 2001, the National Committee on Vital and Health Statistics published the report Information for Health: A Strategy for Building the National Health Information Infrastructure, which outlined a vision for information exchange to improve the health of communities and individuals. This report was widely endorsed and, consequently, All Kids Count staff felt their efforts should be focused on delineating specific recommendations for the future of child health information systems versus duplicating the national efforts.

As the momentum for the national information infrastructure initiative (NHII) grew over the next two years, the staff felt the time was ripe to proceed with plans to host an invitational conference aimed at fostering development of a shared vision for integrated systems. The staff formed a planning committee with representatives from other organizations and held what was informally called the "vision conference" on December 3–4, 2003, at the Emory Conference Center in Atlanta. Seventeen agencies and organizations agreed to support the conference as nonfunding cosponsors. (See Appendix 3 for the list.)

HRSA: A Key Partnership
The project staff worked closely with the Genetic Services Branch of HRSA's Maternal and Child Health Bureau. A mission of the Genetic Services Branch is to facilitate the early identification of individuals with genetic conditions and integrate them into systems of care that are comprehensive, accessible, available, affordable, acceptable, population and community-based, culturally appropriate and family-centered. In 1999, HRSA/MCHB began a grant initiative to facilitate program integration and further infrastructure development at the state level around newborn screening and genetics. The purpose of these state-based programs is the early identification of conditions for which early and timely interventions can lead to the elimination or reduction of associated mortality, morbidity and disability. The system must ensure that the results of newborn screens are shared with the child's medical home for long-term follow-up and care.

During the term of the grant for the technical resource center, the task force received two HRSA contracts worth a total of $875,000 to support HRSA/MCHB efforts in information integration. For example, task force staff identified key elements of a successful integrated information systems project and reported on best practices of seven HRSA/MCHB-funded integration projects on the key elements. Also, HRSA/MCHB and the All Kids Count technical resource center jointly convened a workgroup to identify the principles, core functions and performance measures that an integrated system should be able to perform.

To house the All Kids Count technical resource center activities supported by RWJF and the complementary activities undertaken for HRSA/MCHB, the task force formed an internal umbrella component called the Center for Innovation in Health Information Systems. The same staff members did the work for both All Kids Count and HRSA/MCHB. (Also, with funding from CDC, the same staff helped the Association for State and Territorial Health Officers develop a community of practice to share best practices among states related to emergency response and bioterrorism planning.

In 2002, the task force received additional RJWF funding to support expanded efforts to upgrade the public health information infrastructure (ID#s 043745, 046025 and 046085). At that time, the task force started the Public Health Informatics Institute, and the All Kids Count technical resource center became a program within the institute. (See After the Grant for additional information about the institute.)

Challenges and Changes
To pursue the issue of information privacy, the staff attended conferences on privacy law and funded a Connections member (Rhode Island Department of Health) to produce a white paper assessing that state's data system's compliance with privacy standards. However, the evolving nature of federal confidentiality requirements prevented development of a comprehensive set of privacy guidelines as the project proposal had contemplated.

The Health Insurance Portability and Accountability Act of 1996 (HIPAA) mandated adoption of federal privacy protections for certain health information. In 2000 and again in 2002, DHHS issued HIPAA regulations detailing specific privacy requirements. This unsettled legal and policy landscape, with uncertainty over the interpretation of HIPAA regulations and their effect on individual state privacy requirements, made setting confidentiality guidelines impractical.

Also, after initial study, the technical resource center staff concluded that the field of information integration was too new and full of unknowns to document reliable cost and benefit estimates for child health data. Consequently, the staff postponed the effort to produce a comprehensive cost-benefit analysis, although one of the Connections grantees (Maine Department of Health and Human Services) conducted a cost study of its integration project. The institute is currently developing the business case for integrating child health information systems.

Early in the technical resource center grant, the staff hired a contractor to develop a computer tool — named Catalyst — to help public health organizations formulate information systems strategy. However, the work required cost and outcome data that were not readily available. Given that difficulty, the expense of the undertaking and a determination that the tool was a low priority for health care organizations, the staff discontinued the development effort in the project's second year.

The September 11, 2001, terrorist attacks on New York and Washington and the subsequent anthrax scare also had an impact on the work of the technical resource center. In response to the threat of bioterrorism, public health agencies stepped up their preparedness efforts, causing some to divert resources and personnel from other activities, including information integration.

In summary, national initiatives and events overtook some aspects of the project plans, says Alan R. Hinman, M.D., M.P.H., principal investigator on the All Kids Count technical resource center staff.

 Back to the Table of Contents


RESULTS

In interviews conducted for this report, a report from the technical resource center staff to RWJF and articles published in a supplement to the November 2004 issue of the Journal of Public Health Management and Practice, the staff identified the following among the project's accomplishments and conclusions:

  • The technical resource center strengthened the public health sector's involvement in and commitment to the development of integrated child health information systems. The project staff cited the co-sponsorship of the December 2003 conference by 17 agencies and organizations as evidence of widespread support for information integration. The conference and other project efforts also helped advance a framework for integrating child health information systems. For example, staff members organized and facilitated a HRSA/MCHB-funded work group that developed a set of 19 principles and 22 core functions for integrated child health information systems. (For an in-depth look at the work group's results, see "Principles and Core Functions of Integrated Child Health Information Systems" in the journal supplement, November 2004, available online.)
  • The concept of integrating immunization registries with other child health information systems gained increased support from federal agencies — including HRSA, CDC and CMS. At least in part because of the work of the technical resource center and the partnerships it created, funding for the construction of "silo" systems is certain to become obsolete.
  • The technical resource center established a community of practice dedicated to improving public health information networking. Connections successfully brought together geographically diverse public health agencies to address a complex issue through peer-to-peer learning and demonstrated that collaboration can be an effective model for delivering technical assistance. After the RWJF grant ended, Connections continued with HRSA funding.
  • The technical assistance funds provided to Connections members stimulated solutions to networking issues that were shared with the health community at large. The majority of Connections members reported progress in their integration efforts, although a lack of standard metrics made it impossible for the All Kids Count technical resource center staff to measure their progress in a systematic way. (See the article, "All Kids Count 1991–2004: Developing Information Systems to Improve Child Health and the Delivery of Immunizations and Preventive Services," in the Journal of Public Health Management and Practice, November Supplement, 2004, online.) Through various communications methods, the technical resource center widely disseminated the Connections members' solutions and lessons. (See Communications for details.)
  • The technical resource center helped move immunization registries and integrated child health information systems beyond the concept stage. The challenge now is how to implement and use them effectively. "We conclude the final phase of All Kids Count [the technical resource center] knowing that integrated child health information systems can be developed despite their complexity," the staff wrote in the supplement article cited above. However, demonstrating the role that information systems can play in improving child health outcomes is in its early stages. Considerable "work is still needed to develop a policy case for child health information systems that supports their long-term sustainability," the article says.
  • The technical resource center helped stimulate creation of the Public Health Informatics Institute and laid the foundation for ongoing collaborative efforts to improve health care information technology. Recommendations made at the "vision" conference helped frame the objectives of additional RWJF-funded work carried out by the institute. (See After the Grant.)

Recommendations

In an article in the November 2004 journal supplement, the project's top staff members identified four "common threads" in recommendations made by participants at the December 2003 "vision" conference. The authors (Ross, Saarlas and Hinman) emphasized that the recommendations were by individuals and had not been officially endorsed by any of the 17 agencies and organizations that cosponsored the conference.

The four common-thread recommendations identified by the staff were:

  1. Develop a national coalition of stakeholders to promote integration of child health information systems. The promotional effort should be in the context of ongoing national initiatives to move the nation's health system to electronic record keeping.
  2. Develop the business and policy cases for integrated child information systems.
  3. Develop agreement on standards for collecting and transferring information.
  4. Disseminate information explaining the importance of integrating separate child health information systems in order to improve health and health systems.

(For a full discussion of the conference recommendations, see "A Vision for Child Health Information Systems: Developing Child Health Information Systems to Meet Medical Care and Public Health Needs" in the November 2004 supplement online.)

Communications

The technical resource center staff used a variety of means to disseminate the results of their work. Foremost was support for a journal supplement devoted to child health data integration. The 104-page supplement to the November 2004 issue of the Journal of Public Health Management and Practice contained 16 articles on activities, results and lessons from All Kids Count, related work supported by HRSA/MCHB and state and local information efforts. Some 2,500 copies of the supplement went to journal subscribers and others, and the full text was also available on the Public Health Informatics Institute Web site.

The staff also published two volumes describing the 16 state and local integration projects undertaken by members of Connections. Both volumes — entitled Creating a Road Map: Sharing Knowledge about Integrating Child Health Information Systems — were accessible on the project Web site and the institute site.

In cooperation with HRSA/MCHB, the project published a "sourcebook" for organizations interested in integrating newborn screening data with other child health information that documented best practices and lessons from seven HRSA/MCHB-funded integration projects. A companion tool for assessment and planning enabled agencies to assess their readiness for integrating information systems.

The December 2003 "vision conference" was another key communications effort. The two-day invitational meeting drew about 100 people representing 55 government agencies and private organizations. Richard Shiffman, M.D., M.C.I.S., associate professor of pediatrics at the Yale School of Medicine and associate director of the Yale Center for Medical Informatics, delivered the keynote address. Fourteen additional speakers — including staff members from the technical resource center, HRSA and state and local public health institutions — discussed information integration from various perspectives. Participants divided into breakout sessions to consider and make recommendations in specific areas.

Early in the life of the technical resource center, the staff targeted its communications activities to inform stakeholders of the expansion of All Kids Count from immunization registries to information networking. The technical resource center commissioned a new All Kids Count logo and graphic identity and developed and disseminated a brochure explaining the need for integrated child health information systems. Another brochure explained the purpose of Connections, and the staff also published a quarterly Connections newsletter about activities of the collaborative and its members.

Over the course of the project, the staff issued several news releases, including one identifying the first-round recipients of technical assistance grants. Staff members made numerous presentations to health organizations about the project and information integration issues.

The All Kids Count Web site, which is still accessible, includes information about the purpose of the technical resource center as well as many of the project documents. Information, much of it duplicative, is also available at the Institute site.

(See Bibliography for details of the technical resource center's major dissemination efforts.)

 Back to the Table of Contents


EVALUATION

Although RWJF itself did not evaluate the technical resource center, the staff used a portion of the RWJF grant to hire a consultant, Silver Creek Associates, in Alexandria, Va., to assess the effectiveness of Connections. The two-member evaluation team interviewed 26 Connections participants, conducted a Web-based survey of the members and examined written records. The firm's January 30, 2004, report included the following findings and recommendations:

  • Without exception, all participants believed they had reaped significant benefits from involvement in Connections — benefits for their projects and for themselves as professionals. The most dramatic benefits came from hosting a site visit from technical resource center staff, but members also gained from exposure to the experiences of the other groups.
  • The single most important element of Connections was that it created a community culture of safety, trust, valued relationships and a commitment to mutual support. Another strength was its diversity — bringing together representatives of professional communities that otherwise had limited opportunities to understand each other's perspectives.
  • Some participants, however, believed Connections did not reach its full potential. The evaluation team found some participants who believed (1) the site-meeting segments devoted to general integration issues were not as valuable as the segments that focused on the hosts' specific projects and (2) there was "a loss of community energy and interaction between meetings."
  • The Connections staff should broaden the collaborative's group-learning repertoire beyond site meetings and group projects. Site visits are expensive and group projects may not engage the whole community. The staff should develop various types of events, processes and tools that meet a range of learning objectives.
  • Participants were critical of the ConnectionsZone Web site, and thought the Connections staff should explore additional technology options available to support events and interactions. Conferencing systems superior to the software platform on which the ConnectionsZone interactive site was built are available at a more competitive price. (The staff subsequently switched to another interactive Web technology and also hired Silver Creek Associates to help improve "virtual meeting" practices and techniques.)

(For a full discussion of the evaluation findings, see "All Kids Count Connections: A Community of Practice on Integrating Child Health Information Systems" in the November 2004 supplement online.)

 Back to the Table of Contents


LESSONS LEARNED

In "All Kids Count 1991–2004: Developing Information Systems to Improve Child Health and the Delivery of Immunizations and Preventive Services" in the November 2004 supplement, program staff members said the combined phases of All Kids Count provided several general lessons for health information projects, including:

  1. Recognize the complexity of establishing a population-based information system. Clinical information systems deal with a selective group of people — for example, patients admitted to a particular hospital. Population-based systems, on the other hand, must include everyone who lives in an area and, thus, are more challenging to develop. However, population-based systems produce long-term benefits — such as identifying high-risk groups — and are worth the additional effort.
  2. Develop the business case for information systems. Cost and benefit information is necessary to convince policymakers to invest in child health information systems.
  3. Define the requirements of a new information system to meet users' needs. All Kids Count staff found projects that adopted the motto "If we build it, they will come" were more likely to fail. Successful registry projects surveyed their users' needs and adopted a structure that fit the physician's daily work flow.
  4. Develop information systems according to current standards. One barrier to integrated information systems is the mistaken belief that a given registry's situation is so unique that standards do not apply to its development.
  5. Address common problems collaboratively. Camaraderie and trust are key factors in successful collaborations. The All Kids Count staff deliberately refrained from being judgmental and tried to establish a noncompetitive atmosphere.
  6. Plan for change. Because information systems development is complex and takes 10 years or more, public health agencies should expect significant changes in technology, funding and political priorities. Change is hard but can be managed effectively if the project leadership is aware of the magnitude of change and committed to long-term success.
  7. Plan boldly but build incrementally. An incremental approach permits the demonstration of completed phases, and each success increases the likelihood of a subsequent success. It also permits the project to adapt to inevitable changes in environment and technology.
  8. Develop a good communications strategy. A good communications strategy begins with listening to the various stakeholders and understanding their concerns and needs before shaping informational messages. Although most parents and providers support the notion that health coverage can be improved through better information, they often become confused or anxious when they hear that a government agency is developing a system to "track" their children's health information.
  9. Use the Information. There is a natural tendency to want to ensure all data in a system are complete and accurate before using the system. The downside to this quest for perfection is the risk that the data will not be released for use to physicians.

 Back to the Table of Contents


AFTER THE GRANT

The September 11, 2001, terrorist attacks on New York and Washington and subsequent anthrax incidents focused increased attention on bioterrorism and the ability of public health agencies to detect and react to disease outbreaks.

In response, RWJF in 2002 awarded the task force a $2.87-million, 23-month grant (ID# 046026) to undertake additional activities aimed at upgrading the public health surveillance and information infrastructure, including improving the information management systems used by public health laboratories.

As noted previously, the task force created the Public Health Informatics Institute to perform this work simultaneously with the All Kids Count technical resource center. In May 2004, as the grant for the technical resource center was nearing its end, RWJF awarded the task force a $3.2-million, two-year grant (ID# 046085) to continue the efforts of the Public Health Informatics Institute.

The new grant called for the institute to use its collaborative model to:

  1. Help public health laboratories design a common laboratory information management system.
  2. Assist key stakeholders in constructing a business case for integration of child health information systems.
  3. Convene stakeholders to develop an agenda identifying public health information infrastructure priorities.

The institute's other funders include HRSA/MCHB, the Association of Public Health Laboratories and the Minnesota Department of Health. Connections continues with HRSA/MCHB support, its membership enlarged to include additional federal grantees.

 Back to the Table of Contents


GRANT DETAILS & CONTACT INFORMATION

Project

All Kids Count: Technical Resource Center to Foster Development of Integrated Preventive Health Information Systems

Grantee

Task Force for Child Survival and Development (Decatur,  GA)

  • Amount: $ 4,999,270
    Dates: August 2000 to July 2004
    ID#:  039374

Contact

David A. Ross, Sc.D.
(404) 687-5634
dross@taskforce.org
Kristin Saarlas, M.P.H.
(404) 687-5617
ksaarlas@taskforce.org

Web Site

http://www.allkidscount.org
http://www.phii.org

 Back to the Table of Contents


APPENDICES


Appendix 1

The Definition and Potential Benefits of Integrated Child Health Information Systems

Integrated child health information systems consolidate data on multiple health care services delivered to children and make the information useful to families, medical providers and public health authorities. Integrated systems are accessible through a convenient point of access, ensure optimal health care service delivery and follow-up and support population health assessment.

Integration projects now underway focus on the integration of information systems of four programmatic areas: immunizations; newborn dried blood spot screening; early hearing detection and intervention; and vital registration, as well as others.

For public health officials, integrated systems:

  • Provide information to identify pockets of need, target interventions and resources and evaluate programs.
  • Provide information on coverage and screening follow-up rates.
  • Promote reminder and recall of children who need services and screening.
  • Facilitate changes in schedules for screening and follow-up.

For parents and providers, these systems:

  • Consolidate in one record all health services received by a child.
  • Provide an accurate official copy of a child's health history for entry to day care, school and/or camp.
  • Provide reminders when follow-up care is needed or when follow-up care has been missed.
  • Help ensure timely screening and follow-up care for children whose families move or change health care providers.
  • Prevent unnecessary, duplicative services.


Appendix 2

Results of Projects Supported by the All Kids Count Technical Resource Center

The All Kids Count technical resource center provided 16 technical assistance "mini-grants" to support development of integrated preventive health information systems. Following are brief descriptions of each project as reported in Volumes One and Two of Creating a Road Map: Sharing Knowledge about Integrating Child Health Information Systems.

The first nine projects received funding during 2001–2002 in the initial round of grants and are the subject of Volume One. The other seven projects received funding in 2003 in a second round and are covered in Volume Two.

Round One Projects

1. CalOptima
Project title: Recruitment and Retention Plan for CalOptima's Login Information Network for Kids

This Medicaid managed care organization in California's Orange County evaluated pilot sites using a Web-based children information network (LINK) and developed recruitment plans to expand the network's use by the larger pediatric provider community. Pilot site staff and physicians who were interviewed all viewed LINK as a positive resource but some expressed concerns about the availability of data and ease of use of the system. Project staff concluded, for both pilot and newly recruited sites, physicians and staff members need adequate training, support and awareness of the information available on the system.

2. Kansas Integrated Public Health System
Project title: Childhood Lead Poisoning Prevention Information System Support for a Collaborative Childhood Lead Poisoning Prevention Program

The Kansas project planned to define requirements for an information system to address lead poisoning prevention. The Sedgwick County Health Department was the project sponsor; other partners included the Kansas Department of Health and Environment, the Women, Infant and Children's Program, the state preschool association and the Kansas City Housing Authority. Project staff was not able to complete the project as envisioned because of staff turnover and other competing priorities for the collaboration. Project staff instead began to define the business requirements for an information system that would support all community level public health activities. Project staff reported that an important lesson learned was that the role and expectations of the sponsor in a collaborative effort must be understood.

3. Maine Bureau of Health
Project title: Development of the Business Case Analysis for the Maine Integrated Public Health Information System

The Maine agency contracted with the Maine Health Research Institute to conduct a business case analysis of the agency's integrated information system. The project resulted in a business model, a funding plan and a draft of uniform core data elements and business rules for the system. In the next project phase, staff planned to use these products to recruit the leadership of the Bureau of Health and other community stakeholders to the vision of an integrated public health information system and the possible benefits of this re-engineering.

4. Michigan Department of Community Health
Project title: Envisioning Real-Time Access to Child Health Data by Providers

The agency initiated a systematic process to identify specific child health programs that can be linked with the Michigan Childhood Immunization Registry. The project evaluated the importance, interest and impediments to the potential linkages and developed a plan for eventually integrating these disparate data resources. The evaluation concluded that health data must serve three distinct functions within the system: client services (real-time access and reporting on services provided), health surveillance (epidemiological research on outbreaks of communicable disease, patterns of chronic illness, vital statistics and registries) and administration (managing Medicaid funds and enrollment and preventing fraud and abuse). Project staff noted the need for improvements to the data collected, including standardization and quality, and the creation of a patient identifier.

5. Missouri Department of Health and Senior Services
Project title: Increasing the Quantity and Timeliness of Immunization and Other Reportable Conditions Data in Missouri

The agency worked with a billing data clearinghouse to determine the feasibility of abstracting immunization and other public health reportable conditions data from billing data submitted by health care providers. Preliminary results indicated data abstraction from billing claims provides timely information and is a viable option for obtaining information on vaccine doses provided to clients. Project staff recommended expanding electronic file submissions to include communicable diseases and other reportable conditions.

6. New York City Department of Health and Mental Hygiene
Project title: A Pilot Data Exchange between the Master Child Index and the New York State Medicaid Database

The city agency piloted a data exchange between the city's Master Child Index database and the state Medicaid Management Information System. The pilot project demonstrated the feasibility of ongoing data exchanges with the state system. Staff reported lessons learned included: (1) allow sufficient amounts of time to negotiate between public health agencies and gain program approval and legal consent for data involving Medicaid; (2) pursue and obtain support from people able to influence the Medicaid data exchange process along; and (3) before integrating databases, conduct a pilot.

7. Oregon Department of Human Services
Project title: Communication Plan for FamilyNet, Oregon's Consolidated Health Data System

The Oregon agency created a marketing plan to communicate the benefits of the state's child health data system to health care providers and families. The project identified key stakeholders in the system: health care providers, parents and parent advocacy groups and medical associations. Results of focus groups with parents and providers revealed that for parents the main concerns of a health data system were confidentiality and misuse of data; providers' concerns were the responsibility for gathering and maintaining data and the need to explain the data gathering to parents. Staff produced the creative components of a marketing effort to address perceived barriers and highlight benefits. Elements included endorsements, public relations education and advertising to reach the two audiences of parents and providers.

8. Rhode Island Department of Health
Project title: Confidentiality: The Challenge of Time, Growth and Change

Agency staff assessed its data confidentiality, security and access policies for compliance with federal and state laws and standards. The agency revised some of its processes and materials related to data confidentiality and created a model document for data sharing among programs affiliated with KIDSNET, the statewide integrated child health information system.

9. Utah Department of Health
Project title: Developing Child Health Advanced Records Management (CHARM)

The agency created a data stewardship policy for intra-agency, program-to-program data sharing in connection with CHARM, a state initiative to manage online, real-time communication and data sharing among information systems.

Round Two Projects

10. Kansas Integrated Public Health System
Project title: The Data Used in Your Secondary Use Application May Not Represent What You Think They Do

This project studied data elements in the Kansas integrated system and the secondary use of the data for public health decision-making. The objective was to create a classification scheme for data sources and identify appropriate and inappropriate data usage. In Volume Two of Creating a Road Map: Sharing Knowledge about Integrating Child Health Information Systems, project staff concluded, "…the dream of a comprehensive, internally consistent set of data from disparate sources gathered into a secondary use environment in order to accurately measure the pulse of public health activity, effectiveness and efficiency is going to be a far more difficult undertaking than we initially believed."

11. Missouri Department of Health and Senior Services
Project title: Expansion of Missouri's Integrated Public Health Information System to Include Electronic Blood Lead Level Results: A Pilot Project for Electronic Reporting of Laboratory Blood Lead Level Results

The agency worked with public and private laboratories to increase the electronic laboratory reporting of blood lead level testing results for children 12–72 months of age. The agency reported at the end of the project that much of the preparatory work necessary to implement the electronic reporting improvement had been completed.

12. New York City Department of Health and Mental Hygiene
Project title: Developing and Implementing a Batch Query Application

The agency developed and implemented an application that queried the city child health database and disseminated immunization and lead test information. The application increased the number of records returned to requesting providers and decreased the time spent by the agency on record reviews.

13. Oregon Department of Human Services
Project title: Using Nationally Accepted Standards to Exchange Data between an Immunization Registry and a Managed Care Organization

The agency's immunization registry implemented a synchronized data exchange that populated a Kaiser Permanente Northwest computerized system with information on patients up to age five. This prototype for exchanging data across different systems provided a model applicable to a planned data exchange between the immunization registry and FamilyNet, Oregon's health data system.

14. Utah Department of Health
Project title: Improving Data Quality through Electronic Data Capture

The agency hired a consulting firm to identify the procedures utilized at birthing hospitals to enter the birth record number (BRN) — a unique identifier of newborns initiated by the agency in 2002 — and to make recommendations to ensure data accuracy and completeness. (The BRN had proved beneficial in identifying unregistered births, missed hearing screenings and missed follow-up opportunities, but data entry errors and omissions had reduced the BRN's effectiveness.) Project staff concluded that the effectiveness of unique BRNs could be enhanced by (1) employing electronic data capture technology to enter the BRN, (2) enhancing the systems used for newborn screening and creating the birth registration and (3) improving operational processes to eliminate manual or redundant activities.

15. A group project involving seven organizations: Kansas Integrated Public Health System, Maine Bureau of Health, Missouri Department of Health and Senior Services, New York City Department of Health and Mental Hygiene, Oregon Department of Human Services, Rhode Island Department of Health and Utah Department Health
Project title: De-Duplication Technology and Practices for Integrated Child Health Information Systems.

De-duplication is the term for data cleaning activities performed to match and merge records appropriately. This study examined de-duplication software and approaches, performed limited testing and technical analysis of a number of products and compared their effectiveness, cost and other factors. The study did not identify one single best product but provided a framework to examine alternatives and trade-offs when choosing de-duplication products and strategies to meet integration requirements.

16. A group project involving six organizations: Maine Bureau of Health, Michigan Department of Community Health, New York City Department of Health and Mental Hygiene, Oregon Department of Human Services, Rhode Island Department of Health and Utah Department of Health.
Project title: All Kids Count Mobile Registry Application: A PDA-based Interface to Immunization Registries

A consulting firm worked with the above six organizations to design a personal digital assistant (PDA) product to accommodate the needs of a diverse set of registries. A PDA is a handheld device that combines computing, the Internet and other capabilities. The design team developed architecture and end-user, security and server integration requirements for this product.


Appendix 3

Cosponsors

Cosponsors of "A Vision for Child Health Information Systems: Developing Child Health Information Systems to Meet Medical Care and Public Health Needs," held December 3–4, 2003

  • American Immunization Registry Association
  • Alliance of Community Health Plans
  • Association for Health Center Affiliated Health Plans
  • Agency for Healthcare Research and Quality
  • American Academy of Pediatrics
  • Association of Maternal and Child Health Programs
  • Association of State and Territorial Health Officials
  • Centers for Disease Control and Prevention
  • Commonwealth Fund
  • Family Voices
  • Health Resources and Services Administration
  • March of Dimes
  • National Association of County and City Health Officials
  • National Association of School Nurses
  • National Health Information Infrastructure
  • National Initiative for Children's Healthcare Quality
  • U.S. Department of Agriculture Food and Nutrition Service

 Back to the Table of Contents


BIBLIOGRAPHY

(Current as of date of this report; as provided by grantee organization; not verified by RWJF; items not available from RWJF.)

Book Chapters

Ross DA, Hinman AR, Saarlas K and Foege W. "Foreword." In Public Health Informatics and Information Systems, O'Carroll PW, Yasnoff WA, Ward ME, Ripp LH and Martin EL (eds.). New York: Springer, 2003.

Articles

Journal of Public Health Management and Practice. November Supplement, 2004. The supplement consists of 16 articles describing activities, results and lessons learned from the All Kids Count III project, related work supported by the Health Resources and Services Administration, and state and local efforts to integrate child health information systems. The articles, which are also available online, are:

  • Biondich PG and Grannis SJ. "The Indiana Network for Patient Care: An Integrated Clinical Information System Informed by Over Thirty Years of Experience."
  • Fehrenbach SN, Kelly JCR and Vu C. "Integration of Child Health Information Systems: Current State and Local Health Department Efforts."
  • Hastings TM. "Family Perspectives on Integrated Child Health Information Systems."
  • Hinman AR, Atkinson D, Diehn TN, Eichwald J, Heberer J, Hoyle T, King P, Kossack RE, Williams DC and Zimmerman A. "Principles and Core Functions of Integrated Child Health Information Systems."
  • Hinman AR, Saarlas KN and Lloyd-Puryear MA. "Editorial: Integrating Child Health Information Systems."
  • Hinman AR, Saarlas KN and Ross DA. "A Vision for Child Health Information Systems: Developing Child Health Information Systems to Meet Medical Care and Public Health Needs."
  • Hoyle T and Swanson R. "Assessing What Child Health Information Systems Should Be Integrated: The Michigan Experience."
  • Linzer DS, Lloyd-Puryear MA, Mann M and Kogan MD. "Evolution of a Child Health Profile Initiative."
  • Papadouka V. Schaeffer P, Metroka A, Borthwick A, Tehranifar P, Leighton J, Aponte A, Liao R, Ternier A, Friedman S and Arzt N. "Integrating the New York Citywide Immunization Registry and the Childhood Blood Lead Registry."
  • Ross DA, Hinman AR, Saarlas KN, Lloyd-Puryear MA and Downs SJ. "The Near-Term Future for Child Health Information Systems."
  • Saarlas KN, Hinman AR, Ross DA, Watson Jr. WC, Wild EL, Hastings TM and Richmond PA. "All Kids Count 1991–2004: Developing Information Systems to Improve Child Health and the Delivery of Immunizations and Preventive Services."
  • Shiffman RN. "A Convocation of Wizards: Synergies at the Intersection of Child Health Care, Public Health and Information Technology."
  • Wild EL and Fehrenbach SN. "Assessing Organizational Readiness and Capacity for Developing an Integrated Child Health Information System."
  • Wild EL, Hastings TM, Gubernick R, Ross DA and Fehrenbach SN. "Key Elements for Successful Integrated Health Information Systems: Lessons from the States."
  • Wild EL, Richmond PA, de Merode L and Smith JD. "All Kids Count Connections: A Community of Practice on Integrating Child Health Information Systems."
  • Williams SD and Hollinshead W. "Perspectives on Integrating Child Health Information Systems: Parents, Providers and Public Health."

Hinman AR. "Public Health Information Systems: From Promise to Practice." Journal of Public Health Management and Practice, 8(3): 54–58, 2002.

Hinman AR. "Tracking Immunization." Pediatric Annals, 33(9): 609–615, 2004. Abstract available online.

Horne PR, Saarlas KN and Hinman AR. "Costs of Immunization Registries: Experiences from the All Kids Count II Project." American Journal of Preventive Medicine, 19(2): 94–98, 2000. Abstract available online.

Horne PR, Saarlas KN and Hinman AR. "Update on Immunization Registries." American Journal of Preventive Medicine, 20(2): 174, 2001.

Ross DA. "Implications of Practicing Informatics in the Public Health Sector." Journal of Public Health Management and Practice, 8(3): v–vi, 2002.

Saarlas KN, Edwards KC, Wild E and Richmond P. "Developing Performance Measures for Immunization Registries." Journal of Public Health Management and Practice, 9(1): 47–57, 2003. Abstract available online.

Reports

All Kids Count. (Brochure) Atlanta: Public Health Informatics Institute, 2001.

All Kids Count and Public Health Informatics Institute. (Brochure) Atlanta: Public Health Informatics Institute, 2003.

All Kids Count Connections. (Brochure). Atlanta: Public Health Informatics Institute, 2001.

All Kids Count Connections Evaluation. Alexandria, Va.: Silver Creek Associates, 2004.

All Kids Count Vision Conference Proceedings. Decatur: Ga.: Public Health Informatics Institute, 2004. Available online.

Case Study: Developing Performance Measures for Immunization Registries. Atlanta: Public Health Informatics Institute, 2004. Available online.

Creating a Road Map: Sharing Knowledge about Integrating Child Health Information Systems, Volume One. Decatur, Ga.: Public Health Informatics Institute, 2003. Available online.

Creating a Road Map: Sharing Knowledge about Integrating Child Health Information Systems, Volume Two. Decatur, Ga.: Public Health Informatics Institute, 2004. Available online.

Current State of Integration Survey. Decatur, Ga.: Public Health Informatics Institute, 2003.

Hinman AR and Lloyd-Puryear MA. Integrating Child Health Information Systems. Proceedings of the 2004 Newborn Screening and Generic Testing Symposium. Atlanta: 2004.

Integration of Newborn Screening and Genetic Service Systems with Other Maternal and Child Health Systems: A Sourcebook for Planning and Development. Decatur, Ga.: Public Health Informatics Institute, 2003. Available online.

Integration of Newborn Screening and Genetic Service Systems with Other Maternal and Child Health Systems: A Tool for Assessment and Planning. Decatur, Ga.: Public Health Informatics Institute, 2003. Available online.

Topics in Public Health Informatics: Integrated Child Health Information Systems. Atlanta: Public Health Informatics Institute, 2004. Available online.

Audio-Visuals and Computer Software

Connections, Integrated Child Health Information Systems Community of Practice, a CD-ROM containing visual aids on the information integration initiatives of the Oregon Department of Human Services. From a presentation at a meeting of the Connections collaborative in Portland, Ore., June 25–27, 2002.

World Wide Web Sites

www.allkidscount.org. The public Web site of the All Kids Count program. Although the program itself has concluded, the site continues and contains information on activities, reports and findings related to the program's Phase III efforts to stimulate development of integrated preventive child health information systems. Atlanta: Public Health Informatics Institute, 2001.

www.ConnectionsZone.org. An interactive Web site for members of Connections and other public health practitioners. The site permits users to exchange ideas and address issues related to the integration of child public health information systems. Atlanta: Public Health Informatics Institute, 2001.

www.phii.org. The Web site of the Public Health Informatics Institute, a component of the Task Force for Child Survival and Development. The Institute seeks to advance the ability of the public health sector to apply information systems strategically to improve the health of communities and individuals. The site includes information about Institute activities and the field of health information systems, including access to reports, publications and other materials produced by the All Kids Count Phase III project. Atlanta: Public Health Informatics Institute, 2002.

 Back to the Table of Contents


Report prepared by: Michael H. Brown
Reviewed by: Janet Heroux
Reviewed by: Molly McKaughan
Program Officer: Susan B. Hassmiller