Cred and Micromex Research are excited to announce our partnership to deliver evidence based and rigorous infrastructure and service reviews and community strategic planning services for local government. As a team we are leaders in local government planning, quantitative and qualitative research, and community engagement. Our collaboration will deliver outcomes across the triple bottom line for local government and communities that are based on proven social, scientific and spatial methodologies and utilising our shared 30+ years of experience.
Find out more about Micromex Research here, and have a read of their great evidence-based tip about the benefits of active opt-out over passive opt-in engagement methods below!
Our team of senior specialists look forward to working together with you on your next project - get in touch at firstname.lastname@example.org.
Micromex Research - Case Study – The Ambivalent Resident and the Active Complainant
There are benefits achieved by using a random sample or participants over conducting research using an existing panel or resident opt in surveys. The following data comes from the same survey conducted concurrently as both a random telephone survey of residents and an opt-in online/postal survey communicated by the council via local newspapers, libraries and an online community panel.
In the above table it is easy to see that, in terms of importance, there is no difference in response between the two methodologies. This indicates, from an attitudinal perspective, that both samples are similar. However, once we look at the responses from a satisfaction perspective, it is clear that those who opt-in to complete an online/postal questionnaire respond much differently to respondents who agreed to participate in an active opt-out CATI methodology.
In the above table we can see that from the broader community telephone measure, 31% of residents are satisfied to very satisfied with this council’s long term planning, compared to only 15% from the online methodology. If we aggregate the not at all satisfied to not very satisfied scores, we can see that 54% of opt-in respondents had a low level of satisfaction with Council’s performance in this area, compared to 31% of residents interviewed via phone.
The same response pattern is reflected in all studies we have conducted using a multi-modal approach.
In the table below we can see the level of dissatisfaction is still significantly different between the methodologies.
In fact, when we look at the distribution of % responses we can see that for the randomly conducted phone survey, respondents were most likely to respond somewhere between ‘somewhat satisfied’ and ‘satisfied’, whereas the opt–in respondents were most likely to respond somewhere between ‘not very satisfied’ and ‘somewhat satisfied’.
In summary, it is clear that while both methodologies provide a similar score with regard to importance, with regard to a satisfaction measure, those who opted in via a passive methodology were significantly more negative than those surveyed by telephone.
As an example, a random sample size of n=400 residents 18 y/o and older provides a maximum sampling error of +/-5% at 95% confidence. This means that if we extended the survey and interviewed all residents 18 y/o and older, that 19 times in 20 we would expect the result to be within +/-5% of the outcome indicated in the phone survey results.
So from a sampling perspective, we can confidently say the results of the telephone survey reflect the general community result.
Þ If you don’t use an active opt-out methodology, you will not achieve an inclusive, accurate or representative satisfaction measure.
The results from the community online survey are also reflective, however, not of the broader community but rather of community members who are likely to opt-in to fill in a survey, or to be highly engaged. Any community survey that relies on an opt-in data collection method is much more likely to obtain results that over represent the extreme.
Þ If you rely on an opt-in passive methodology, you will over represent the extreme.