Case Study - Comparison of classic and ReDem cleaning using the Magenta Smart-City study

In this article, we shed light on the effects of ReDem data cleaning on the interpretation of the results of the Magenta Smart City study.

The study was conducted by TripleM – Matzka Markt- und Meinungsforschung on behalf of Magenta Business and cleaned by ReDem.

In einer Gegenüberstellung von konventioneller- und ReDem-Bereinigung zeigen wir eindrucksvoll, die Auswirkungen der Bereinigung durch ReDem auf die Interpretation der Daten. Besonders bei inhaltlich komplexen Fragestellungen ist die Verwendung der ReDem-Bereinigung von großer Bedeutung für die Ableitung von Handlungsempfehlungen.

Survey design | Description of the survey

For the cleaning of the raw data of the Magenta Smart City study, first the conventional cleaning of speeders, straightliners and also the control of open ended answers, was performed manually in a multi-step procedure by TripleM.

In comparison, the raw data was scored and cleaned using the ReDem Score. The ReDem Score performs all quality checks simultaneously and is then used as the basis for cleaning. The ReDem Score also includes the so-called "ReDem Prediction Score", which measures superficial response behavior through projective control questions. This method identifies respondents whose answers were given carelessly and without cognitive effort. Especially with more complex and longer questions, this effect can lead to negative distortions and can hardly be corrected by conventional cleaning methods.

The following excerpts from the Smart City study show the effect of the ReDem cleaning compared to conventional cleaning.

Influence of the cleaning on social demographics

Social demographics are barely changed by the ReDem cleaning.

Data were socio-demographically weighted and cleaned using ReDem.

Awareness of the term "Smart City"

For the term awareness "Smart City", the ReDem correction brings a slight increase.

Again, no major change was expected after the ReDem cleaning, as this is a relatively simple and clear question.

Importance of community/city becoming smart city?

The importance of Smart City to one's community would be significantly underestimated WITHOUT the ReDem cleaning.

Without the cleaning by ReDem, there would be a differentiated interpretation of the data. The importance increases by over 10 percentage points, to 75,9 percent.
This question was an important element of the entire study and essential for a valid result.

Due to the more complex content of the questions, the effect of the ReDem correction can be seen very clearly.

Assessment of Smart City—agreement with statements

The cleaning by ReDem shows partly significant differences in the assessment of Smart City.

This example also shows that superficial response behaviour is very likely, especially with questions where respondents have to make an increased cognitive effort.

Increasing the quality of the sample

ReDem cleaning increases the quality score (ReDem score) of the whole sample by 14 points to 81 points.

The ReDem score measures data quality, with a value between 0 and 100, before and after cleaning.

Conclusion

In the Magenta Smart City study, superficial response behaviour occurred primarily with more complex questions, with increased cognitive effort.

This example underlines the importance of the ReDem cleaning and shows very well which questions are likely to be biased by superficial answers and which questions/items are less susceptible to it.

Click here for the press release of the study:
https://newsroom.magenta.at/2021/08/05/magenta-smart-city-studie/

Florian Kögl
Florian is the founder and CEO of ReDem®. He is also a board member of the Austrian Market Research Association and has extensive experience in the development of innovative software solutions.