Category Archives: Evaluation
Bonnie Zima, MD, MPH, an alumna of the Robert Wood Johnson Foundation (RWJF) Clinical Scholars program (1989-1991), published a study this month that appeared in a special supplement of Pediatrics with articles by RWJF Clinical Scholars on child health quality. Pediatrics is the official journal of the American Academy of Pediatrics. Zima is a professor-in-residence in child and adolescent psychiatry at the University of California in Los Angeles (UCLA) and associate director of the UCLA Center for Health Services & Society.
Human Capital Blog: Why did you decide to review the new child mental health quality measures?
Bonnie Zima: This paper was written to stimulate discussion about the need for a paradigm shift for quality measurement for children that more closely aligns research with the accelerated pace of quality measure development.
These are exciting times for those who believe that the quality of child health care can be improved through measurement and public reporting. However, this direction also raises questions about how to improve our methods and data infrastructure to monitor the quality of care received in real-time and to link adherence to quality indicators to clinical outcomes that are meaningful to parents, child advocates, providers, agency leaders and policy-makers.
HCB: Why did you focus on child mental health?
Zima: We focused on child mental health care because quality measurement poses additional challenges that can be used as a stimulus to improve future measure development.
Some of the areas for future research include development of a stronger evidence base to support nationally recommended care processes in community-based populations; models of care coordination across multiple care sectors that often have discrete funding streams, such as specialty mental health, public health, education, child welfare, and juvenile justice; and the development of interventions that more flexibly align service delivery with children’s clinical needs, especially for those with co-morbid mental and physical health conditions.
Lori Melichar, PhD, is a senior program officer at the Robert Wood Johnson Foundation (RWJF).
On February 12, the Robert Wood Johnson Foundation launched a new website that can serve as a long-awaited repository for work we have funded over the last 10 years that invests in advancing the science of quality improvement (QI) research and evaluation. We hope the website also provides the opportunity for researchers and other health care professionals engaged in QI work to access resources and to connect with colleagues with mutual interests.
The launch coincided with a virtual meeting on Advancing the Science of QI Research and Evaluation (ASQUIRE). The group convened to hear findings from grantees of the Foundation’s Evaluating QI Training Programs Initiative (PQI).
Meeting participants were tasked with thinking about how the website can best disseminate their work as well as collect, house and spread tools, frameworks, methods and models to assist those doing QI and those evaluating QI efforts. Grantees were joined by experts in QI research, practice and evaluation and a lively discussion (sometimes a debate) ensued.
The grantees of the Robert Wood Johnson Foundation (RWJF) conduct pioneering and influential research that makes remarkable contributions to the field, and to the research and evaluation community at large. Each year, RWJF identifies the 20 most popular pieces of research on its website, and invites the public to vote for the five Most Influential Research Articles.
After a record-breaking vote – the most since the poll was launched in 2008 – the 2012 “Final 5” have been announced.
“One of the most important things that bring all of these winners together is that they’ve hooked a research issue to a social issue that’s much bigger than the article itself,” RWJF Vice President of Research and Evaluation, David C. Colby, PhD, said in a Q&A about the finalists. “They helped solve a piece of that puzzle for people.”
One of the “Final 5,” a study on physician wages in states with expanded scope of practice for advanced practice registered nurses (APRNs), was funded by the RWJF Initiative on the Future of Nursing. The study found that physician wages are not depressed when APRNs practice independently.
“In 2010, the Institute of Medicine report on the future of nursing recommended that advanced practice registered nurses should be able to practice to the full extent of their education and training,” co-author Patricia Pittman, PhD, said in an interview. “As we looked at the issue as researchers, we scratched our heads and tried to understand what the dangers were for primary care physicians, and whether there were economic interests at stake… We viewed this study as a first step towards building an evidence base around the economics of these reforms.”
In addition to reading their research, this year you can read personal interviews with the grantees, where they share their motivation for the studies, key findings, examples of visibility and influence, and lessons that might be valuable to other researchers who want to maximize the impact and reach of their work.
Paul Glassman, DDS, MA, MBA, is director of the Dental Pipeline National Learning Institute, a program of the Robert Wood Johnson Foundation. Glassman is a professor of dental practice and director of community oral health at the University of the Pacific, Arthur A. Dugoni School of Dentistry in San Francisco.
I recently had the opportunity to visit the British Royal Observatory in Greenwich, UK, current home of John Harrison’s famous clocks, which provided the solution to one of the most vexing problems in 17th and 18th Century Europe. As eloquently chronicled in Dava Sobel’s book Longitude: The True Story of a Lone Genius Who Solved the Greatest Scientific Problem of His Time, the 17th and 18th Century naval fleets of the world were plagued by the inability to accurately measure longitude. A ship’s captain at sea could get very precise readings of the ship’s latitude by measuring the angle between the sun at noon and the horizon. However, measurement of longitude required knowing the current time at a known point, such as London, which would allow the captain to compare the position of stars as seen from the ship, to where they would have been at the known point at that precise time.
Unfortunately, timepieces of that day were too inaccurate to facilitate these measurements. As a result, inefficient routes were followed to increase safety, many ships ran aground anyway, lives were lost, and the economic consequences for the shipping industry were staggering. In 1714 the British Parliament offered the “Longitude Prize” of £20,000 for a solution to this problem. It was not until 1772, after many attempts and failures, that Harrison was awarded this prize for his 4th timekeeper, a clock that could keep accurate time aboard a moving ship, and Parliament declared that the problem had been solved. This development allowed the British naval fleet to obtain world dominance at the end of the 18th Century.
The oral health system in our country has its own longitude problem. Our inability to accurately measure where we are and chart a course forward has tremendous human and economic consequences.
The New York Academy of Medicine is the National Program Office for the Robert Wood Johnson Foundation Health & Society Scholars program, which works to reduce population health disparities and improve the health of all Americans. The New York Academy recently conducted a survey of 17 thought leaders in primary care and population health. In the fourth of five blog posts, we share a synthesis of what those leaders had to say. All quotes are printed with permission.
The Role of Primary Care and Clinical Incentives: Most discussants agreed that primary care and the patient-centered medical home (PCMH) movement are important foundations upon which to build broad-based population health activities. While acknowledging that these systems are already over-burdened with clinical responsibilities—and that current incentive structures are poorly aligned to accomplish this goal—many cited the degree of overlap in the missions of primary care and public health institutions as a starting point.
Community Care of North Carolina (CCNC), for example, evolved slowly and steadily over 25 years from a clinical quality network to a statewide multi-sectoral public-private partnership based on the PCMH. Allen Dobson, MD, described the key components as: the formation of cooperative provider networks; introduction of population management tools; case management; and data infrastructure with rapid feedback to providers. “Community Care is bottom-up and physician-led with respect to quality improvement…but because the collaboration includes the public health department, we are also looking at population metrics regardless of whether or not we’re managing that population.” Financing is organized using flexible per-member-per-month allotments that allow networks to put resources into quality measurement. Dobson cited external evaluations demonstrating that the overall project is cost-saving, with CCNC responsible for nearly $1.5 billion in lower costs from 2007-09.
You’re Invited to a Virtual Meeting on Advancing the Science of Quality Improvement Research and Evaluation
Lori Melichar, PhD, is a senior program officer at the Robert Wood Johnson Foundation (RWJF).
The Robert Wood Johnson Foundation’s mission is to improve the health and health care of all Americans. In pursuit of this mission, we seek to improve the quality of care provided in hospitals, ambulatory care centers, public health departments, and other settings where health is enhanced or health care is delivered.
Within the past 15 years, Quality Improvement (QI)—the process-based data-driven approach to improving the quality of a product or service through iterative action-evaluation cycles—has emerged as a promising strategy to accomplish this goal, and RWJF funded several national programs to “demonstrate” the potential of QI to improve health care processes, staff engagement and patient outcomes. The Foundation’s Pursuing Perfection Program, which had as its goal to help hospital and physician organizations improve patient outcomes dramatically by pursuing perfection in major care processes, employed QI tools such as Plan-Do-Study-Act cycles and improvement collaboratives to accomplish this goal. Another program, Transforming Care at the Bedside, taught frontline nurses the skills and methods of QI and empowered these staff to engage in activities to transform hospital care. Paths to Recovery is an RWJF program that used QI processes to improve the systems of care that provided substance abuse treatment. Aligning Forces for Quality is RWJF’s signature effort to lift the overall quality of health care in targeted communities, reduce racial and ethnic disparities, and provide models for national reform.
The New York Academy of Medicine is the National Program Office for the Robert Wood Johnson Foundation Health & Society Scholars program, which works to reduce population health disparities and improve the health of all Americans. The New York Academy recently conducted a survey of 17 thought leaders in primary care and population health. In the third of five blog posts, we share a synthesis of what those leaders had to say. All quotes are printed with permission.
Local and State Health Department Collaboration: Most discussants agreed that clinical systems and health departments use different notions of ‘population’—and historically are not well integrated. As David Stevens, MD, noted, “There’s capacity that needs to be built on a common language on how to work together that isn’t there because they've been separated so long.”
There was, however, a prevailing notion that this dynamic is changing in important ways. Many cited the convergence of IRS Community Health Needs Assessment (CHNA) requirements and new public health accreditation standards as a potential blueprint for future collaborations. Clinical delivery systems, generally well-resourced but with limited community assessment and intervention skills, are now responsible for conducting a CHNA every three years while developing and implementing an action plan to address identified needs.
The New York Academy of Medicine is the National Program Office for the Robert Wood Johnson Foundation Health & Society Scholars program, which works to reduce population health disparities and improve the health of all Americans. The New York Academy recently conducted a survey of 17 thought leaders in primary care and population health. In the second of five blog posts, we share a synthesis of what those leaders had to say. All quotes are printed with permission.
Challenges in the Urban Context: Discussants converged upon care fragmentation and community diversity as the most difficult challenges associated with working in urban settings. There may be enormous heterogeneity within populations in urban areas with respect to racial, ethnic, and sociodemographic characteristics. Subgroups may vary with regard to exposures, behaviors, and values. The sense of community that can be essential to leveraging social groups may not necessarily be present or uniform throughout a geographic area, necessitating multiple tailored communication strategies. Even between cities, there is significant heterogeneity, such that non-clinical interventions may be less transferable than, say, a chronic disease model.
Communities that do exist may not necessarily conform to geographic boundaries, and the geopolitical boundaries and layers of jurisdiction in place may mean little to those communities. This changes how confident clinical systems can be for outreach and aspects of care that might reach beyond the office, and in general it can be particularly challenging to know what services are being provided for a patient, where, and by whom. This accountability problem makes it easier for high-risk patients to fall through the cracks.
The New York Academy of Medicine is the National Program Office for the Robert Wood Johnson Foundation Health & Society Scholars program, which works to reduce population health disparities and improve the health of all Americans. The New York Academy recently conducted a survey of 17 thought leaders* in primary care and population health. In the first of five blog posts, we share a synthesis of what those leaders had to say. All quotes are printed with permission.
Defining Population Health: Many discussants cited the definition of population health developed by David Kindig, MD, PhD, as a reference point: “health outcomes of a group of individuals, including the distribution of such outcomes within the group.” Regardless of specific vantage point, there was a generally shared sentiment that population health should be thought of broadly and in common terms by a range of clinical and non-clinical stakeholders.
More discussants described a baseline framework of a clinical delivery system oriented around patients in a practice, in contrast with a public health system oriented around geographic communities. A more clinical, or “population medicine,” perspective often centered around evidence-based interventions and disease management categories so as to triage and allocate health care resources in a cost-effective manner.
Long working hours that cause fatigue, sleepiness, burnout and depression are a threat to the personal safety of medical residents, according to a Mayo Clinic study published this month. Working conditions associated with these characteristics are linked to motor vehicle crashes and near crashes, and may contribute to exposure to blood and body fluids on the job.
In the survey of 340 internal medicine residents in training at the Mayo Clinic, 168 respondents (56%) reported a motor vehicle incident during their training. Of those incidents, 34 were motor vehicle crashes, and 130 were near misses. Sixty residents reported falling asleep while driving, and 53 reported falling asleep while stopped in traffic. Residents attribute these incidents to diminished quality of life, exhaustion and depression, and fatigue and sleepiness, the study finds.
Residents also reported exposure to blood and body fluid during their training, some of which was attributed to fatigue. The researchers call the rates “reassuringly low,” but caution that “it is not possible to definitively rule out associations of distress with [blood and body fluid] exposure.”
“These findings indicate that resident distress is related not only to patient safety and quality of care but to residents’ personal safety as well,” the study says. “In addition to ongoing efforts to limit physician fatigue and sleepiness, interventions to promote well-being and reduce distress among physicians are needed to improve both patient and resident safety.”