Are Lower Response Rates Hazardous to Your Health Survey?

An Analysis of Three State Telephone Health Surveys

Surveying more people rather than blindly pursuing response rates may be a smarter tactic for researchers.

Telephone surveys are invaluable sources of data for health services researchers. To achieve high response rates (70% is considered good) and counter the possibility that people who do not respond or refuse to participate in a survey are somehow different and would change the results (nonresponse bias), survey organizations can employ numerous aggressive and costly tactics. For example, using costs provided by one survey center, a survey completed after four calls cost $42.40. A survey completed with the most aggressive attempts (up to 50 calls over the course of months), including using a specially trained person to convince a person who initially refused to participate, cost $98.90.

The researchers of this study applied the above cost structure and three hypothetical follow-up protocols to three different state health surveys to determine if there would be differences in key health variables being measured. They found that while response rates understandably were lower under less aggressive contact scenarios, the key variables were not different.

Rather than spending money on expensive, aggressive follow-ups, researchers could increase the sample size—more than double it—and use other strategies, such as auxiliary information available on responders and nonresponders alike, to identify areas where nonresponder bias may occur.