In April 2020, we conducted our two-part validation study together with Telemark Marketing. In cooperation with Telemark - Marketing Managing Director MMag. Robert Sobotka, our goal was to test the quality improvement of survey results with ReDem.
The hypothesis to be tested was as follows:
The results after cleaning and weighting by ReDem show a higher level of match with the reference data compared to the raw data.
As reference data, we used data collected through a 50/50 CATI/CAWI survey by Telemark Marketing, with an cleaned sample of 8000 (Austrian population). Since we only used an uncleaned sample of 200, through Google Surveys, for this study, we assume an existing difference in data quality (sample size, survey method). With the use of ReDem, it should now be possible to improve the data quality and thus generate a result that is closer to the reference data.
In the first part of the study, participants were asked about their experience with Austrian banks. As mentioned above, Google Surveys was used as the data source.
Baseline of the validation study
- Data source: Google Surveys
- Sample size: 200 (Austrian population)
- Reference data: 50/50 CATI/CAWI survey by Telemark Marketing, n~8000 (representative for Austrian population)
The result at a glance:
Frage: How satisfied are you with your bank's performance overall? What grade would you give them?
Blue = Non-cleaned & unweighted result (each answer has equal weight) Green = Result after ReDem, Yellow = Reference data CATI/CAWI survey n~8000, Figures in percent (%)
As it can be well seen in the result, the error rate was reduced for every single answer option. Overall, the improvement was 37.2%.
Specifically, the improvement in response option 1 (Very Good) was 36%, in response option 2 (Good) 43%, in response option 3 (Satisfactory) 32.9%, in response option 4 (less Satisfactory) 39.9% and in response option 5 (Not Satisfactory) 34.2%.
Our hypothesis that the "lower quality data" would move towards the reference data by using ReDem and thus improve the result could thus be confirmed.
We would like to thank Mr. Sobotka very much for the excellent cooperation. Further results of this two-part validation study can be found in our following article.
