Social desirability is a major challenge for quality in surveys. These responses introduce bias into the data and can negatively affect the quality of recommendations for action. Socially desirable response behavior occurs primarily when respondents answer according to social norms and describes the tendency to answer in a way that is socially "desirable." Socially desirable behavior can occur when the questions are about sensitive or delicate topics. Examples include questions about sustainability, political attitudes, or alcohol consumption. The survey method also has an influence on response behavior. For example, the presence of the interviewer or other persons can increase the effects of social desirability.
Since social desirability is a psychological behavior, this bias is difficult to identify..
In a case study, together with the Austrian Gallup Institute, we were able to show how the "ReDem-Social-Desirability-Score" can be used to automatically identify and correct social desirability bias.
How can social desirability be corrected with the ReDem?
In short, the ReDem Social Desirability Score (SDS) makes it possible to identify social desirability in quantitative surveys. Our algorithm is designed in such a way that the application is as simple as possible and runs automatically. The SDS, like other ReDem quality scores, uses information from projective control questions. In this process, respondents have to make an assessment of the response behavior of others. However, the SDS does not assess individual response behavior, but rather collective response behavior. This means that the SDS is not suitable as a criterion for cleaning at respondent level, but as a weighting of response options.
The SDS is based on the psychological phenomenon of the false consensus effect, which we have already highlighted in a previous article.
The false consensus effect is an unconscious psychological effect with the following premises:
- People who hold a certain opinion or attitude estimate it to be more widespread than it actually is.
- People who hold the opposite opinion are more likely to underestimate the frequency of other opinions.
Thus, if respondents do not answer honestly, the projections will most likely not match the conditions of the false consensus effect.
The average of all available projections is then used to evaluate a respondent in order to ensure a reliable result.
Study setup
In our pilot study with the Austrian Gallup Institute questions were specifically chosen in which socially desirable response behavior was expected.
The goal was to detect and correct the influence of social desirability with the help of the SDS. Specifically, the proportion of socially undesirable response options was to be increased.
- Study 1: n = 500 representative of the Austrian population
- Study 2: n = 1000 representative of the Austrian population
- Online interviews by Gallup's own online panel
The following examples represent an excerpt of the study and highlight the correction effect of the SDS on the outcome of the questions. The correction was made using the ReDem weighting method. The result without correction is represented by the blue bars and the corrected result by the green bars.
Study 1 - Question 1
What is your personal position on the issue of migration, on refugees in Austria?
In this example about the opinions on migration, one can see the correction in the socially undesirable answers very well. Answer options 3 and 4, which tend to express a negative attitude towards refugees, were corrected significantly upwards by the SDS.
Study 1 - Question 2
How much do you personally pay attention to a healthy and balanced diet?
The question on healthy and balanced nutrition also shows a correction in the socially undesirable answer option. The attitude that little attention is paid to healthy eating becomes significantly more relevant due to the correction of the SDS.
Study 2 - Question 1
When shopping, how important is it to you that you buy sustainably and, above all, in an environmentally friendly way?
A similar effect was also found in the second study. This question was intended to shed light on purchasing behaviour with regard to sustainability and environmental friendliness. Here too, the proportion of the socially undesirable answer that sustainability and environmental friendliness are only used for marketing purposes increased significantly with the help of the SDS.
Conclusion
In this case study, it was impressively demonstrated in all examples how it is possible with the help of the Social Desirability Score to bring about a correction of the social desirability bias and thus improve the quality of the results.
A great advantage is also the convenience of use in practice. Only standardized projective control questions need to be added to the questionnaire, according to which the algorithm automatically performs the calculation.
We would like to thank the Austrian Gallup Institute for their excellent cooperation.
If you have any questions about this study or would like to test ReDem, please contact us at office@redem.io
