The Program Being Evaluated

Over the past five years, e-health, the use of information technology in the health field, has been increasingly recognized as an essential tool for improving quality and reducing costs among health care organizations and employers. As a greater number of consumers, patients and providers use e-health applications, there is also a greater need for more rigorous evaluation of the quality of e-health programs. In response to this need, RWJF launched Health e-Technologies Initiative: Building the Science of e-Health (HETI). The goals of HETI were to expand the body of knowledge about the efficacy, costs, cost effectiveness and overall quality of e-health applications currently in use for health behavior change and chronic disease management, and to expand the body of knowledge about how to evaluate, compare and improve them.

About the Evaluation

Led by Seth Emont, Ph.D., M.S., of White Mountain Research Associates LLC, the goals of the assessment were to:

• assess the contribution of HETI to the field of e-health research for behavior change and chronic disease management;
• identify and explore options for follow-on program activities and resources that would help to sustain and spread the contribution that HETI has made to the field; and
• explore the potential role of HETI in supporting and informing the Foundation’s current team strategies.

Phase I of the assessment addressed HETI’s unique niche and brand value in the field, potential field-building activities, the most promising strategies to assure that the initiative’s products will have maximal impact on the field, and networking activities.

Phase II focused on a small group of stakeholders to explore the “value added” of e-health tools in engaging consumers and improving the quality of health care delivery.

Summary of Methods

The methods included interviews and sector-specific surveys of opinion leaders in the field of e-health, including people from academic settings, commercial settings, government agencies, and philanthropy, and of key stakeholders in e-health, including health care providers and health plan administrators and consumers.

Knowledge and Impact

Major findings generated from the interviews and surveys with opinion leaders and stakeholders in e-health include the following:

• Many of the experts emphasized that there is a strong need to improve the evidence base around e-health applications through more rigorous research and evaluation of e-health and Web-based technologies.
• A number of experts recommended that HETI continue to harvest the findings from its grantees, as well as to create syntheses from these studies for use by various stakeholders (e.g., researchers, consumers, providers, policy-makers, potential funders and e-health partners).
• HETI can be a powerful voice across a number of stakeholders, serving as an “honest broker,” thereby, encouraging collaborative (not competitive) efforts for targeting e-health research and interventions.
• HETI can play a role by examining reimbursement issues, return on investment and cost-benefit analysis for e-health applications (where it is also critical to bring insurance companies into the fold).
• HETI could play a prominent role in generating a set of basic definitions and measurable standards around e-health in the development of applications and Web sites.
• E-health can be integrated into existing and future RWJF initiatives. For example, opportunities to fund small development grants—and not large research grants—would allow health care systems and researchers to refine e-health products and provide scientific support.
• A suite of e-health tools focusing on decision support, behavior change, health care tools and personal health records would support the field and develop relationships across partners.