February 2002

Grant Results

SUMMARY

From 1992 to 1997, staff from the Maryland Health Care Access and Cost Commission developed and fielded a survey to measure Medicaid recipients' satisfaction with their medical care.

Key Results

  • In 1995, project staff mailed a 67-item questionnaire—the Maryland Medicaid Recipient Survey—to a random sample of 3,000 Medicaid recipients.

    The survey measured Medicaid recipients' perceptions of care and access to services they received in two settings:
    • An experimental Medicaid program called Maryland Access to Care.
    • Traditional health maintenance organizations (HMOs).

Key Findings

  • Generally, respondents rated HMOs less favorably than the Maryland Access to Care program.
    • HMO enrollees gave worse ratings than Maryland Access to Care enrollee respondents to all three summary items in the survey:
      • Overall rating of care.
      • Evaluation of how many things needed to be improved.
      • Whether the respondent would recommend a provider to someone with a similar health condition.
    • HMO respondents also gave worse ratings on nine of 16 access-to-care issues, including:
      • Choice of doctor or nurse practitioner.
      • Length of wait for an appointment.
      • Convenience of location.
    • HMO respondents gave worse ratings on all questions related to interpersonal treatment by office staff and evaluation of the physician or nurse practitioner.
    • HMO respondents rated out-of-pocket expenses as better.

Funding
The Robert Wood Johnson Foundation (RWJF) supported this project through a grant of $109,943.

 See Grant Detail & Contact Information
 Back to the Table of Contents


THE PROJECT

With its Medicaid costs rising steeply, the state of Maryland was granted a federal waiver in 1991 to test Maryland Access to Care (MAC), an experimental model that incorporated some of the elements of managed care—including having patients choose a primary physician—but that retained a fee-for-service structure.

The plan, intended to shift resources toward primary and preventive care, offered physicians and clinics increased reimbursement for seeing Medicaid patients in exchange for taking on additional duties, including making referrals to specialists and staffing a telephone line that patients could call for medical advice. MAC was also expected to benefit patients by giving them a primary doctor for the family and to cut costs by reducing emergency visits.

The aim of this grant from RWJF to the Maryland Health Care Access and Cost Commission, an advisory group of the Maryland Department of Health and Mental Hygiene, was the design and testing of a reliable survey instrument that would measure Medicaid recipients' perceptions of care and access to services they received in two settings: (1) an experimental Medicaid program known as MAC and (2) HMOs.

Under a subcontract from the state, investigators in the Johns Hopkins Program for Medical Technology and Practice Assessment developed and fielded the Maryland Medicaid Recipient Survey. Findings were used to compare the perceptions of enrollees served by MAC and those served by HMOs.

The state of Maryland was able to supplement the RWJF grant with Medicaid matching funds. The federal match was 45 percent through June 1995 and increased to 50 percent for the remainder for the project. In-kind services were donated by the state and by Johns Hopkins University School of Medicine faculty.

 Back to the Table of Contents


RESULTS

  • The investigators tested four survey methods and identified a cost-effective and reliable way to survey Medicaid recipients. After assessing several methods of locating eligible respondents—who may move frequently, be homeless, or not have phones—the investigators settled on a mail-in, self-administered questionnaire, which had a 31 percent response rate. While random-digit dialing targeting telephone exchanges in areas with a high Medicaid population yielded the highest response rate—43 percent—cost factors made that approach impractical.
  • A 67-item questionnaire was mailed to a random sample of 3,000 Medicaid recipients, aged 18 to 64, whose addresses were collected from recent claims. A follow-up mailing was sent about three weeks later. Of the 926 who responded, 416 were enrolled in MAC and 160 were in Medicaid HMOs. The remainder was not enrolled in managed care, but received services on a fee-for-service basis.
  • The reliability and validity of the survey data were assessed and confirmed.
  • Generally, respondents rated HMOs less favorably than the MAC program.
    • HMO enrollees who responded gave worse ratings than MAC enrollee respondents to all three summary items in the survey: overall rating of care, evaluation of how many things needed to be improved, and whether the respondent would recommend a provider to someone with a similar health condition.
    • HMO respondents also gave worse ratings on 9 of 16 access-to-care issues: choice of doctor or nurse practitioner, length of wait for an appointment, convenience of location, waiting time at the office or clinic, ability to reach the office or clinic by telephone both during and after office hours, availability of the same provider at each visit, ease of obtaining needed referrals to other doctors, ease of getting into the hospital when needed, and ease of getting needed medicine. Ratings of HMOs and MACs were similar on most other access-related questions such as hours the clinic or offices are open, appearance and comfort of the office or clinic, how well the buildings and office rooms met respondents' needs (e.g., bathrooms and handicapped access); ease of changing doctor or nurse practitioner; ease of getting a referral to another doctor; and procedure for signing up to continue Medical Assistance (Medicaid).
    • HMO respondents gave worse ratings on all questions related to interpersonal treatment by office staff and evaluation of the physician (MD) or nurse practitioner (NP): for example, how well the MD or NP explained things, amount of time the MD or NP spent with the respondent, how well the MD or NP listened and paid attention, and helpfulness of treatment prescribed by the MD or NP.
    • HMO respondents rated out-of-pocket expenses as better.
  • According to the principal investigators, the methodology used in this survey proved useful for measuring the satisfaction of Medicaid recipients and has contributed to a number of successful national efforts to survey Medicaid recipients.

Communications

The survey instrument and findings were presented at professional forums of the American Public Health Association, the Association for Health Services Research, the National Association of Primary Care, and elsewhere. The grantee organization produced a booklet and an information packet on the survey and responded to multiple requests for the material from researchers in other states and organizations interested in surveying Medicaid recipients. An article is forthcoming in the Journal of Public Health Management Practice. (See the Bibliography for details.)

 Back to the Table of Contents


AFTER THE GRANT

Maryland replaced the experimental MAC program with a capitated system of Medicaid managed care in which the state contracts with various HMOs to provide care. Survey findings informed the design of the new system. The Maryland Department of Health and Hygiene continues to send out an information packet about the survey.

Because the Rand Corporation chose Maryland as a demonstration site for another survey instrument—its Consumer Assessment of Health Plans Survey—there are no immediate plans to continue administering the Maryland Medicaid Recipients Survey. The principal investigator, who also works with Rand, indicated that several questions developed under the RWJF grant have been incorporated into the Rand survey.

 Back to the Table of Contents


GRANT DETAILS & CONTACT INFORMATION

Project

Survey of Access and Patient Satisfaction of Maryland Medicaid Clients

Grantee

State of Maryland Health Care Access and Cost Commission (Baltimore,  MD)

  • Amount: $ 109,943
    Dates: March 1992 to June 1997
    ID#:  019107

Contact

Mary E. Stuart, Sc.D.
(410) 455-2084
Haya R. Rubin, M.D., Ph.D.
(410) 955-9870
Hrubin@jhmi.edu

 Back to the Table of Contents


BIBLIOGRAPHY

(Current as of date of this report; as provided by grantee organization; not verified by RWJF; items not available from RWJF.)

Articles

Yurk R, Jenckes MW, Stuart ME, Schaffer TJ, Lockwood RS, Das A and Rubin HR. "Benchmarking Applications: Linking State Strategic Planning, Quality Improvement, and Consumer Reporting." Journal of Public Health Management Practice, 7(3): 47–58, 2001. Abstract available online.

Survey Instruments

"The Maryland Medicaid Recipients' Survey." Maryland Department of Health and Hygiene and the Johns Hopkins Program for Medical Technology and Practice Assessment, fielded July and August 1995.

Presentations and Testimony

L Easter, J O'Brien, Haya R. Rubin, Mollie Jenckes, L Bone and Mary E. Stuart, "Maryland Medicaid Patient Satisfaction Surveys as a Quality Assurance Tool," at the 122nd Annual Meeting of the American Public Health Association, 1994, Washington.

Mary E. Stuart, "Surveying Special Populations: Minorities and Urban/Rural," at the Agency for Health Care Policy and Research/ RWJF conference "Consumer Survey Information in a Reformed Health Care System," September 1994, Vienna, VA.

Mary E. Stuart, "Maryland Medicaid Patient Satisfaction Surveys as a Quality Assurance Tool," at the annual meeting of the American Public Health Association, October 1994, Washington.

Mollie Jenckes, "Maryland Medicaid Recipients Ratings of Pediatric Care by HMOs and Fee-for-Service Providers," at the annual meeting of the American Public Health Association, October 1994, Washington.

Ann M. Kerns, "Developing an Accountability System: The Use of Populations Surveys" at the annual program meeting of the Workshop for States of the Information for State Health Policy Program, May 1996 Memphis, TN.

Myron Wickham, "Measuring Medicaid Recipients' Satisfaction Levels in the Maryland Medicaid HealthChoice Program," May 1996, Baltimore County, MD.

Myron Wickham, "Methods for Monitoring Medicaid Managed Care," at the American Public Health Association Medicaid Study Group, November 1996, New York.

Haya R. Rubin, Mollie Jenckes, Thomas J. Shaffer, Robin Yurk, Mary E. Stuart and Myron Wickham, "Report to the Maryland Department of Health and Mental Hygiene on the Maryland Medicaid Recipient Survey," June 1997, Baltimore.

Mollie Jenckes, "Maryland Medicaid Recipients' Survey: Using Patients' Ratings of Quality and Access to Compare HMOs and Fee-for-Service Providers," poster presentation at the annual meeting of the Association for Health Services Research, June 1997, Chicago.

 Back to the Table of Contents


Report prepared by: Gale Scott
Reviewed by: Karyn Feiden
Reviewed by: Janet Heroux
Program Officer: James R. Knickman

Most Requested