Response biases are a major challenge for the quality of surveys. The different types of response bias are summarized under the term "response bias".
Specifically, the occurrence of response biases systematically distorts the survey results and thus negatively influences them. These distortions ultimately lead to the results deviating from the true opinions or attitudes of the respondents.
When and why do such biases occur in the first place?
Response bias occurs primarily when respondents' answers are influenced not only by the question but also by other factors. These factors can lie with the respondent, in the interview situation as well as in the survey instrument. Reasons for the occurrence of response tendencies can be a superficial answering of the questions, a supposed conformity with social norms or also the formulation of the questions.
What does an optimal response process of a respondent look like?
The following aspects are important for a respondent to answer questions in the best possible way:
- Neutral presentation of the question.
- Understanding the question.
- Extraction of information from memory.
- Assess which of the information is relevant to the answer.
- Uninfluenced indication of the answer.
Each of these steps may involve a high level of cognitive effort, which may lead respondents to answer the question only superficially.
Other types of errors in surveys
Response bias, however, represents only one of many possible types of error that can occur in surveys. Other types of bias or error include incorrect sample selection or interview failure (non-response bias), which can negatively affect the results. You can find out more about the different types of errors in surveys in our article on this topic.
Fig.1 Types of errors in surveys
In the following, we now present the most important manifestations of response biases. These are divided into formal aspects on the one hand and content-related aspects on the other, whereby formal tendencies can often also be traced back to content-related tendencies.
Formal response biases
Formal response biases are biases that occur regardless of the content of the question.
Satisficing
Satisficing is superficial response behavior in which respondents reduce their cognitive effort by giving the first answer that comes to mind. In the case of strong, satisficing, the questions are often no longer read at all and thus the content of the question is no longer perceived. Online surveys are particularly susceptible to this. Satisficing is usually the trigger of response tendencies, such as the "agreement tendency (acquiescence) or the "tendency toward the middle".
Acquiescence
In this type of bias, also called agreement bias or "yes-sayer" bias, survey participants agree regardless of the content of the question or statement. This means that answer options such as "I agree" or "Yes" are given. Various studies indicate that respondents with "low social status" and lower cognitive abilities are particularly prone to acquiescence. The tendency to acquiesce is influenced by the complexity of the survey, the respondent's ability, and by motivation.
Center tendency
Especially in cases of uncertainty, respondents tend to give an answer in the middle (e.g. answer 3 on a Likert scale of 1 to 5). Freely according to the principle "I can do the least wrong there!". Basically, the more uncertain the respondent's assessments - the greater the tendency toward the center. However, satisficing can also increase the tendency toward the center.
Straightlining
"Straightlining" occurs when respondents to a Question matrix or item battery always answer with the same pattern, for example by always giving the first answer choice. This occurs mainly in self-administered surveys (without an interviewer).
While it is relatively easy to identify respondents who, for example, always select the first answer (Fig. 2), it is significantly more difficult to identify respondents who alternate between two or more answer choices (Fig.3).
Fig. 2
Fig. 3
This response behaviour usually occurs when respondents are bored or find the questioning too complex and demanding.
Leniency-Severity bias
The tendency toward leniency or severity describes the tendency of respondents to give extreme answers in rating scales - regardless of the content. Thus, this formal tendency represents the opposite of the tendency toward the center. Studies have shown that telephone and face-to-face interviews are more affected by this response bias than online or paper interviews. The tendency here is toward particularly positive responses, which may be related to the presence of an interviewer.
Response tendencies in terms of content
The following response tendencies, unlike superficial response behaviour, are dependent on the content of the questions, as well as on factors such as how the questions are asked and who asks the questions.
Social desirability
Probably the best-known form of response bias is social desirability. Socially desirable response behaviour occurs primarily when respondents answer according to social norms. Social desirability describes the tendency to answer in a way that is "desired" in society.
Socially desirable behaviour can occur when the questions are about sensitive or delicate topics. Examples are questions about alcohol consumption, political attitudes or sexual behaviour.
The survey method also has an influence on the response behaviour.
The presence of the interviewer or other persons can reinforce the effects of social desirability.
Interviewer effects
Interviewer effects depend on the situation of the interviews. The causes here lie primarily in the behavior, but also in the demographic characteristics of the interviewers. Generally, the presence of an interviewer has positive effects. Follow-up questions can be asked and unclear points clarified to ensure that the question was understood.
However, an interviewer can also have negative effects on the response behavior of the respondents. This is the case, for example, when answer categories are not read out neutrally and respondents are led to give certain answers.
How can these response biases be reduced or offset?
Questionnaire design
A very important aspect is the wording of the questions. Here, it is particularly important to ensure that the questions are formulated neutrally so that the respondents are not steered in a predetermined direction.
Neutral interviewers
When conducting surveys with an interviewer, particular care must be taken to ensure that the questions are asked as neutrally as possible. In order to avoid a tendency to agree, attention should also be paid to the pace at which the questions are asked.
The use of ReDem
One solution for identifying and correcting response bias is our data quality solution.
Our tool analyzes the response behavior of respondents, using specific projective control questions. This allows us to identify some of the response biases mentioned above, such as social desirability.
