Biases in the response behaviour of survey participants. What are they and how can they be compensated?

Response bias is a major challenge to the quality of surveys. The different types of response bias are summarised under the term "response bias".

Specifically, the occurrence of response biases systematically distorts and negatively influences survey results. These distortions ultimately lead to the results deviating from the true opinions or attitudes of the respondents.

When and why do such distortions occur in the first place?

Response bias occurs primarily when respondents' answers are influenced not only by the question but also by other factors. These factors can lie with the respondent, in the interview situation as well as in the survey instrument. Reasons for the occurrence of response tendencies can be a superficial answering of the questions, a supposed conformity with social norms or also the formulation of the questions.

What does an optimal response process of a respondent look like?

The following aspects are important for a respondent to answer questions in the best possible way:

  1. Neutral presentation of the question.
  2. Understanding the question.
  3. Extraction of information from memory.
  4. Assess which of the information is relevant to the answer.
  5. Uninfluenced indication of the answer.

Each of these steps can involve a high level of cognitive effort, which can lead respondents to answer the question only superficially.

Other types of survey errors

Response bias, however, represents only one of many possible types of error that can occur in surveys. Other types of bias or error are, for example, incorrect sample selection or interview failure (non-response bias), which can negatively influence the result. You can find more about the different types of errors in surveys in our contribution to the topic

Fig.1 Types of errors in surveys

In the following, we will now present the most important manifestations of response tendencies. These are divided into formal aspects on the one hand and content-related aspects on the other, whereby formal tendencies can usually also be traced back to content-related tendencies.

Formal response tendencies

Formal response tendencies are biases that occur regardless of the content of the question. Most formal response tendencies are based on superficial response behaviour (satisficing).

Satisficing

So-called "satisficing" is superficial response behaviour in which respondents reduce their cognitive effort by giving the first answer that comes to mind. With strong satisficing behaviour, the questions are often no longer read at all and thus the content of the question is no longer perceived. Online surveys are particularly susceptible to this. Satisficing is usually the trigger of response tendencies, such as the "tendency to agree (Akquieszenz)" or the "tendency towards the middle".

Akquiescence

In this type of bias, also called agreement tendency or "yes-man" tendency, respondents agree regardless of the content of the question. This means that answer options such as "I agree" or "Yes" are given. Various studies indicate that respondents with low social status and lower cognitive abilities are particularly prone to akquiescence. The tendency towards acquiescence, like other forms of satisficing, is influenced by the complexity of the question, the respondent's ability and his or her motivation.

Tendency towards the centre

In case of uncertainty, respondents tend to give an answer in the middle (e.g. answer 3 on a Likert scale from 1 to 5). Freely according to the motto "I can do the least wrong there! Basically, the more uncertain the respondent's assessments, the greater the tendency towards the middle. But satisficing can also increase the tendency towards the middle.

Straightlining

"Straightlining" occurs when respondents to a Question matrix or item battery always answer with the same pattern, for example by always giving the first answer choice. This occurs mainly in self-administered surveys (without an interviewer).
While it is relatively easy to identify respondents who, for example, always select the first answer (Fig. 2), it is significantly more difficult to identify respondents who alternate between two or more answer choices (Fig.3).

Fig. 2

Fig. 3

This response behaviour usually occurs when respondents are bored or find the questioning too complex and demanding.

Tendency towards leniency or harshness

The tendency towards mildness or harshness describes the tendency of respondents to give extreme answers in rating scales - regardless of the content. Thus, this formal tendency is the opposite of the tendency towards the middle. Studies have shown that telephone and face-to-face interviews are more affected by this response tendency than online or written interviews. However, the tendency here is towards particularly positive answers, which may be related to the presence of an interviewer.

 

Response tendencies in terms of content

The following response tendencies, unlike superficial response behaviour, are dependent on the content of the questions, as well as on factors such as how the questions are asked and who asks the questions.

Social desirability

Probably the best-known form of response bias is social desirability. Socially desirable response behaviour occurs primarily when respondents answer according to social norms. Social desirability describes the tendency to answer in a way that is "desired" in society.
Socially desirable behaviour can occur when the questions are about sensitive or delicate topics. Examples are questions about alcohol consumption, political attitudes or sexual behaviour.
The survey method also has an influence on the response behaviour.
The presence of the interviewer or other persons can reinforce the effects of social desirability.

Interviewer effects

Interviewer effects depend on the situation of the interviews. The causes here lie primarily in the behaviour, but also in the demographic characteristics of the interviewers. Basically, the presence of an interviewer has positive effects. He can ask follow-up questions and clarify unclear points to ensure that the respondents have understood the question.
However, an interviewer can also show behaviour that can have a negative impact on the respondent's answering behaviour. This is the case, for example, when answer categories are not read out neutrally and the interviewee is led to give certain answers.

 

How can these response biases be reduced or compensated for?

Questionnaire design

A very important aspect is the wording of the questions. Here it is important to ensure that the questions are formulated neutrally so that the respondents are not steered in a certain direction.

Neutral interviewers

When interviewing with an interviewer, special care should be taken to ensure that the questions are asked as neutrally as possible. In order to avoid a tendency to agree, attention should also be paid to the pace at which the questions are asked.

The use of redem

One solution for identifying and correcting response bias is our method-independent, data-cleansing software.
Our technology analyses the response behaviour of respondents using special projective control questions and can detect some of the response tendencies listed above. Psychological characteristics in the responses can be used to assess the response quality of each respondent.
Following this procedure, each participant receives an individual weighting, depending on the quality of the answer, with which existing biases in the result can be corrected.