Unlikely tools for unlikely times
9 things to consider before using IVR as a research tool
By Ruth Canagarajah, Samuel Oyegunle, Leonard Waweru and Elijah Baraza
In 2016, INC ran an interview titled “This CEO runs a billion-dollar company with no offices or email.” The interview profiled Matt Mullenweg, the CEO of Automattic, which runs WordPress. WordPress operates over 35% of websites on the internet and has been run purely remotely since long before a global pandemic showed the rest of the world the possibilities created by remote work.
At Busara, the shift to remote work forced us to think critically about how we approach our operations — particularly research. We developed the Busara online toolkit that allowed us to pivot our operations and still offer high quality behavioral insights to our clients. As part of this shift, we turned to Interactive Voice Response technology. IVR was appealing because it works on the most basic phones, meaning we could reach respondents wherever they are.
IVR technology also has a number of advantages over interviewer-administered modes of telephone data collection such as:
- It eliminates interviewer bias through consistency: Since a pre-recorded voice is employed to administer all survey items, IVR participants are read questions, response options, and instructions in the exact same way.
- Convenience: Respondents can choose to dial in or be called at a particular time by the system making data collection very convenient.
- Local Language: Designing an IVR system that speaks the language of the local users enables more effective interaction with the system.
- Nimble Feedback Mechanism: Surveys using IVR for market research are more cost-effective and nimble compared to traditional methods of conducting research. Hence, IVR surveys can be highly useful to evaluate the feedback and implement the necessary changes.
We put together 9 things we learned from working with IVR as a tool for remote research for you to consider before you embark on an IVR project.
- Consider the most effective logistics for your study.
Researchers need to consider if they are using a one-way or two-way system, what costs will be incurred by callers and the languages on the system that callers can select into. We also advise presenting the IVR number via a written method such as text.
2. Multiple dry runs are required for an IVR to be successful
Having a team of people engaging with the IVR platform in the early stages to get a sense of the respondents’ experiences will go a long way in making sure your instrument is research ready.
3. Respondents should be trained on engaging with IVR platforms
A thorough training for respondents on what to expect and which options to choose for specific responses would be crucial to avoid confusion while engaging an IVR platform. Such training would also be important in raising awareness on the importance of inputting actionable data, as well as carefully guiding the respondents on how to input correct responses.
4. IVR suffers from high attrition rates
The absence of human interaction in IVR could consequently lead to high drop rates. This could happen for many reasons — respondents could be predisposed, disinterested or uncomfortable answering certain questions. However, when follow up calls were made to some of the respondents, they were comfortable to provide responses, implying that the lack of human interaction could cause attrition in IVR studies. This was echoed in a study conducted to assess the feasibility of IVR in providing self-care services among patients. If you want to still avoid human interaction altogether, make sure your IVR approach limits the questions to only the essentials.
5. IVR requires automatic data checks to limit data entry issues
During our IVR study, we noticed a number of respondents who either mistyped responses, or input abnormal figures on sensitive questions touching on income per month, income expectations and number of children. These issues led to a lot of time and effort being spent on data cleaning in order to report correct averages and trends. For future IVR studies, it would be important to create automatic data checks to prevent respondents from typing unexpected responses.
6. Constant behavioral nudges increase interactions with IVR
For one-way IVR, respondents are required to contact the platform in order to take part in a survey. In our IVR study, it was noted that sending constant gain-framed reminders to the respondents increased interactions with the IVR platform. It is important to emphasize that all questions should be answered in order for a respondent to be remunerated.
7. Respondents should know the amount of airtime required before engaging an IVR platform
Some of the respondents that were reached during our follow up indicated that they dropped off due to lack of airtime. The project team consequently sent messages indicating how much airtime was required for each participant before engaging the IVR platform and this increased interactions with the IVR platform.
8. A feature that enables repetition of questions/instructions is required for IVR
Participants may need to listen to some questions/instructions more than once due to background disruptions or for clarity. It is important to have a number that a respondent can press for a particular question/instruction to be repeated.
9. IVR can be costly to run if the number of respondents reaching the platform are not controlled for
While snowball sampling might be crucial in increasing interactions, the amount of airtime spent in an IVR exercise can be too high if the number of respondents reaching the system are not capped during budgeting. It would therefore be ideal to recruit a specific number of respondents fitting a given selection criteria, who will then be allowed to interact with the IVR to minimize airtime usage.
IVR technology might still need some looking into before it can be the remote hero our time needs, but our research shows that it might be worth looking into — especially when trying to reach respondents who have limited access to smartphones.
