Case Study - Social desirability correction

Socially desirable responses are a major challenge for quality in surveys. These dishonest answers lead to distortions in the data and can negatively influence the quality of recommendations for action. Socially desirable response behaviour occurs primarily when respondents answer according to social norms and describes the tendency to answer in a way that is socially "desirable". Socially desirable behaviour can occur when the questions are about sensitive or delicate topics. Examples of this are questions about sustainability, political attitudes or alcohol consumption. The survey method also has an influence on response behaviour. For example, the presence of the interviewer or other persons can increase the effects of social desirability.

Since social desirability is a psychological behaviour, this bias is difficult to identify through traditional data cleaning methods.

In a case study, together with the Austrian Gallup Institut, we show you how it is possible to identify and correct social desirability bias in an automated way using the "Redem Social Desirability Score".

 

What is the Redem Social Desirability Score (SDS) and how does it work?

In short, the Social Desirability Score (SDS) makes it possible to identify honest answers in quantitative surveys. Our algorithm is designed in such a way that the application in practice is as simple as possible and runs automatically.
The SDS, like other redem quality scores, uses information from projective control questions. This requires respondents to make an assessment of the response behaviour of others.
SDS is based on the psychological phenomenon of the False Consensus Effektswhich we have already highlighted in a previous article.

The false consensus effect is an unconscious psychological effect with the following premises:

  • People who hold a certain opinion or attitude estimate it to be more widespread than it actually is. 
  • People who hold the opposite opinion are more likely to underestimate the frequency of other opinions.

Thus, if a respondent does not answer honestly, the projections will most likely not match the conditions of the false consensus effect.
The average of all available projections is then used to evaluate a respondent in order to ensure a reliable result.

Study setup

In our pilot study with the Austrian Gallup Institute, questions were specifically chosen in which socially desirable response behaviour was to be expected.

The aim was to correct the influence of social desirability with the help of the SDS. Specifically, the proportion of socially undesirable response options was to be increased.

  • Study 1: n = 500 representative of the Austrian population
  • Study 2: n = 1000 representative of the Austrian population
  • Online interviews by Gallup's own online panel

The following examples are an excerpt of the study and highlight the correction influence of the SDS on the outcome of the questions. The correction was made using the Redem weighting method. The result without correction by the Redem-SDS, is shown by the blue bars and the corrected result by the green bars.

Study 1 - Question 1

What is your personal position on the issue of migration, on refugees in Austria?

In this example about the opinion on migration, one can see very nicely the correction in the socially undesirable answers. Answer options 3 and 4, which tend to express a negative attitude towards refugees, were clearly corrected upwards by the SDS.

Study 1 - Question 2

How much do you personally pay attention to a healthy and balanced diet?

The question on healthy and balanced nutrition also shows a correction in the socially undesirable answer option. The attitude that little attention is paid to healthy eating becomes significantly more relevant through the automated correction of the SDS.

Study 2 - Question 1

When shopping, how important is it to you that you buy sustainably and, above all, in an environmentally friendly way?

A similar effect was also found in the second study. This question was intended to shed light on purchasing behaviour with regard to sustainability and environmental friendliness. Here too, the proportion of the socially undesirable answer that sustainability and environmental friendliness are only used for marketing purposes increased significantly with the help of the SDS.

Conclusion

In this case study, it was impressively demonstrated in all examples how it is possible to achieve a correction of the bias caused by social desirability with the help of the social desirability score and thus improve the quality of the results.
A big advantage is also the ease of use in practice. Only standardised projective control questions need to be added to the questionnaire, according to which the Redem software automatically carries out the evaluation of the respondents.

We would like to thank the Austrian Gallup Institut for their excellent cooperation.

If you have any questions about this study or would like to test Redem, please contact us at office@redem.io