The science of medicine is built on a foundation of data. But before data can guide care, it must first be captured and analyzed. When it comes to measuring the quality and effectiveness of pediatric care, a commonly held view is that we don’t gather the data we need, and don’t have good measures for assessing what we gather.
“It’s been a desert,” says Mark A. Schuster, M.D., Ph.D., an alumnus of the Robert Wood Johnson Foundation (RWJF) Clinical Scholars program (1991-1993). “There has been very little investment in quality measures for children, and as a result, there’s a real dearth of well-developed measures…. It’s hard to say what the gaps are, because it feels like the whole field is one large gap.”
Lawrence Kleinman, M.D., M.P.H., also a Clinical Scholars alumnus (1990-1992), agrees. “We’ve not matured to the point as a field where we measure with nuance,” he says, “or measure in areas where there is uncertainty, or where there is not a fixed consensus. And unfortunately, most of children’s health care takes place in those areas.”
The data gaps are evident in the day-to-day practice of medicine, Kleinman explains. “We know, for example, that kids get a lot of ear tubes for reasons that are not consistent with the guidelines for care,” he says. “And we know that the number of tonsillectomies and adenoidectomies has doubled in the last ten years—a huge increase. But we don’t know if children’s health outcomes are better or not as a result, because we don’t measure the outcomes. Clearly, some people think it’s helping, and presumably parents believe it. Maybe they’re right. But we don’t know.”
AHRQ Moves to Fill the Data Gaps
With such concerns in mind, the Agency for Healthcare Research and Quality of the U.S. Department of Health and Human Services has been at work for some time developing an initial set of measures for pediatric care. In late 2009, the agency released for public comment a set of 24 specific quality measures grouped into five general areas: prevention and health promotion, management of acute conditions, management of chronic conditions, the family experience of care and access to care.
Then this spring, AHRQ followed up with a series of grants, funded by 2009’s Children's Health Insurance Program Reauthorization Act (CHIPRA), creating seven separate Pediatric Quality Measures Program Centers of Excellence. Each Center is charged with testing, refining and adding to AHRQ’s draft measures.
Kleinman and Schuster are leaders of two of the seven grantee groups, and three of the remaining five are also led by alumni of the Clinical Scholars program: Gary Freed, M.D., M.P.H., (1990-1992), Rita Mangione-Smith, M.D., M.P.H., (1995-1997) and Jeffrey Silber, M.D., Ph.D., (1983-1985).
Each of the groups will develop new measures as well as improve some of AHRQ’s existing measures. Schuster’s project brings together a number of institutions, including Children’s Hospital Boston, where he is chief of the Division of General Pediatrics, and Harvard Medical School, where he is William Berenberg professor of pediatrics; as well as Harvard School of Public Health, Harvard Pilgrim Health Care, Massachusetts General Hospital, the Massachusetts League of Community Health Centers, the National Initiative for Children’s Healthcare Quality, MassHealth (Massachusetts’ Medicaid program) and others. The project will work with a number of organizations, including health care providers, health centers, physicians organizations, Boston Public Schools and parents organizations. Its first two assignments from AHRQ will be to explore measures related to hospital readmissions and the in-patient family experience of care.
Schuster notes that the first of these measures—readmissions—swirls with controversy. “If you send home a patient who has been admitted with an asthma exacerbation or one who had surgery, and she is subsequently readmitted, it raises concerns about whether she was discharged too early or received care that was otherwise inadequate or inappropriate,” he explains. “Or, it might be that she was ready to go home, but nobody explained to the parents how to administer medicine properly, or that she didn’t receive needed follow-up care with the primary care provider.” He adds, “Much of the concern has to do with the lack of clarity over whom a preventable readmission should be attributed to–the hospital, the patient, the doctor. That isn’t always clear.”
The participating organizations in Kleinman’s Center of Excellence are similarly impressive. The project brings his home institution, Mount Sinai School of Medicine, New York, into partnership with the American Academy of Pediatrics, the American Academy of Family Physicians, the Joint Commission, the National Committee for Quality Assurance, the National Initiative for Children's Healthcare Quality, the Child and Adolescent Health Measurement Initiative, Oregon Health & Science University, Northwestern University, and the New York State Department of Public Health, as well as a number of family and consumer organizations.
AHRQ has tasked Kleinman’s group with developing and refining measures related to the availability of high-risk obstetric care for women, in-patient quality and safety of perinatal care, and the use of emergency rooms for children with asthma. Kleinman notes that the asthma measure will be particularly challenging, because the issue can be viewed through a number of lenses, including the extent to which emergency room visits reflect inadequate primary care for asthma, whether the right medications are being prescribed, whether doctors are communicating well with parents and whether parents are following guidelines given them. “The quality we’re measuring here is typically not about the quality of care children are getting in the emergency room, but rather why they ended up going there,” he says.
The end product of the work of each of the seven new Pediatric Quality Measure Program Centers of Excellence will be a series of measures that have been documented and tested in a variety of settings.
‘This Is Who I’d Choose’
The prominent leadership of alumni of the RWJF Clinical Scholars program in these Centers of Excellence is more than just the luck of the draw. “It reflects the role of the Clinical Scholars program in pediatrics research,” says National Program Director Desmond K. Runyan, M.D., Dr.P.H. “It’s not a particular surprise to me that so many Clinical Scholars are in this group, knowing the people who’ve been in the program and knowing the work they’ve done since. Honestly, if I were thinking about who should do this work, this is who I’d choose.”
Kleinman and Schuster return the compliment, crediting the program with providing them with the training and network of contacts they needed to make a mark on the field. “The training I got as a Clinical Scholar exposed me to the field of quality measurement for the first time,” Schuster says, “and gave me an appreciation for the importance of accurately measuring quality. It gave me a great education in the field and helped create my enthusiasm for doing the work.”
Kleinman concurs. “The Clinical Scholars program is foundational to all of this,” he says. “I could not overstate its importance. It was the basis of my methodological skills. It vastly influenced how I do the work and how I think about the work. And the network has been crucial, too. It provided some of my collaborators and taught me how to develop a network.”
Kleinman is equally enthusiastic about the research that lies ahead. “Honestly, this should be a blast,” he says. “This is one of the largest commitments ever to child health care services research; it’s big science in health care delivery. And it’s an opportunity to improve the quality of care for children through better measurement and the use of standards. By holding people accountable, we can drive improvement. And ultimately, it’s about getting kids healthier. So there’s a tremendous reward for contributing to the process.”
For more than three decades, the Robert Wood Johnson Foundation Clinical Scholars program has fostered the development of physicians who are leading the transformation of health care in the United States through positions in academic medicine, public health and other leadership roles. Through the program, future leaders learn to conduct innovative research and work with communities, organizations, practitioners and policy-makers on issues important to the health and well-being of all Americans.