Computer-assisted telephone interviewing (CATI) starter kit

This guide is an evolving work-in-progress. View this article as a logged in user (if you're not a user now, start an account for free) and click on the "Follow" button in the top right to get updates on changes to this starter kit. This guide has two additional parts, part 2 on the basic case management sample workflow and part 3 on the advanced case management sample workflow.

Phone interviews have some advantages over face-to-face interviews, including cost-effectiveness, amongst others. Importantly, you might be considering computer-assisted telephone interviewing (CATI) as an alternative to computer-assisted personal interviewing (CAPI) as a response to the COVID-19 health crisis. In this guide, we'll direct you to make an informed decision on the approach to CATI that works best for you, given your resources and objectives.

If you prefer, to jump straight to a deployable sample form that you can try now, skip to section 4.

Note: We recommend using the early release version of SurveyCTO Collect for Android if you're implementing CATI, rather than the official release. Many of the improvements we've made will dramatically improve your CATI experience using SurveyCTO. Read on to learn more.

1. Overview

For anyone new to CATI, who has decided to implement this solution and manage it themselves, there are a number of choices to make, and requirements to be aware of. How you approach CATI depends on your resources and current circumstances. In the first part of this guide, we'll aim to answer high level questions to do with executing CATI.

First though, should you be implementing CATI? There are of course other alternatives to face-to-face data collection, but not all of them will be optimal for your project. CATI is still interviewer-mediated data collection, so CATI is a great choice whenever questions require some explaining or where response lists are longer than just a few choices. A well-trained interviewer, whether face-to-face or over the phone, can be an important ingredient in survey responses that are grounded in a good understanding of questions, ultimately leading to higher quality data.

If CATI is a good choice for your project, you'll have to decide what approach works for you. While we feel this guide presents an ideal approach to CATI on SurveyCTO, it is not the only one. Existing projects with little time that are pivoting to CATI might have enumerators working on paper printouts of respondent phone numbers, making manual records of call attempts. Such an approach is not likely to scale well, but it is an option that might buy you some time.

Read on to learn about CATI requirements, adapting instruments for CATI, available sample CATI workflows, CATI data quality, remote training, and data monitoring.

For a video overview on how to get started with CATI, and a look at our sample CATI workflow, see our webinar recording on the subject.

2. Requirements

CATI can be flexibly deployed using SurveyCTO with a few different configurations, but there are some base requirements.

2.1 Contact details

You will of course need phone numbers to call! You may already have a list of study participants, including phone numbers, in which case this requirement has been met. Depending on the research subject, it may be possible to assemble a list of contacts through your own research. Alternatively, there are also third party services that can sell you pre-existing lists of qualifying contacts, or help assemble new lists (common in the marketing industry).

Otherwise, thanks to the ability to generate random numbers inside SurveyCTO, you could also decide to experiment with Random Digit Dialing (RDD). You can also prepare lists of random phone numbers for pre-loading into forms.

2.2 Data capture hardware

Data capture using SurveyCTO can be done using smartphones, tablets, or computers. It is even possible to have remote team members use their own hardware (as opposed to equipment provided by you). There are a few options:

  • Use a single smartphone or tablet that supports a SIM card to both make calls and capture data. Supports automatic number dialing on Android.
  • Use a WiFi-only tablet for data capture, and a separate smartphone or feature phone for calls. Requires manual dialing.
  • Use a laptop or desktop computer for data capture, and a smartphone, feature phone, or Skype on the laptop for calls. Requires manual dialing with a separate phone.

Beyond data capture devices, we suggest that phone interviewers use earphones with in-line microphones. Otherwise, using speakerphone will be necessary for calls when using the same device for calls and data capture. Earphones will also afford more privacy to the respondent.

2.3 Internet service

Your team will need to connect to the internet to submit filled forms, and in the most ideal case, to help to publish form data to a call tracking system for the most ideal CATI system using SurveyCTO. However, CATI on SurveyCTO can work well even without a good internet connection, so long as phone interviewers are able to reliably make calls.

2.4 Third party services (optional)

Depending on how your phone survey works, it might benefit from some third party services:

  • Cross-network mobile credit vending services (like Chura in Kenya), for the distribution of mobile minutes/credit. This can be important for a.) topping up the credit of your phone interviewers, and b.) distributing incentives to respondents. Some of these will have user-friendly spreadsheet upload options for distributing mobile minutes (like Chura), and others will have developer-facing options for programmatic distribution (like Flickswitch in South Africa).
  • SIM card service management systems (like SIMcontrol in South Africa), for managing prepaid mobile phone service for your team of phone interviewers. Setup controls like limits (e.g. like how much internet service can be used), share a common balance, and conveniently manage all your SIM cards from one place.
  • Bulk SMS services (e.g. Bulk SMS or Twilio), for sending SMS messages to lists of respondents. These can be used for providing advanced notice of calls to improve response rates, for handling informed consent ahead of calls, and/or distributing or notifying respondents of incentives.
See our guide on requesting consent via bulk SMS. You can also record consent via SMS in a form.
  • Cloud telephony services (like Exotel) for some advanced and convenient features, like centralized billing (no topping up team members), call masking for security, and platform-independent call-recording options. Requirements like these may take some developer-level expertise to implement.

The above are merely suggestions of types of tools that can be value ads to your approach to CATI. None of these tools are requirements, nor are these endorsements. If you have any suggestions of tools you think might help others adapting to CATI, please share in the user forum.

3. Adapting your instruments

The following is a non-exhaustive list of advice on instrument adaptation:

  • Keep your interviews short to improve response rates. Some sources advise that phone surveys should be about 15 minutes at most, but perhaps with some combination of incentives and buy-in from respondents, you might extend that closer to 30 minutes.
  • Reduce the length of your surveys with the aim of fitting the most important questions into a single phone call. To do this, you'll need to consider which questions are crucial and which questions are less important.
  • Use your understanding of importance to help order your questions from most to least important, so you'll be sure to capture the most essential information.
  • If you're unable to reduce your list of questions down to a length that could be handled in a single phone call, consider splitting your questions over more than one form, each intended for a separate phone call.
  • Plan for things to go wrong, because phone calls get disconnected, and respondents hang up because they lose patience. Design your forms so they can be submitted at any point.

Please do additional research on questionnaire design for phone surveys.

4. Adding your questions into our CATI templates

We have prepared several template forms to help you get started with CATI. Simply delete the mock research questions in one of our sample form templates, and replace them with your own. The templates are designed to take care of the process of tracking attempts to contact respondents, making appointments with them, and limiting how much effort goes into attempting to interview each respondent.

All CATI workflows using SurveyCTO on Android are greatly enhanced through support for launching phone calls directly from SurveyCTO Collect, using the call-phone-number field plug-in (read more about field plug-ins).

You are free to customize these templates as you need to, adding features to account for scenarios we didn't account for in our samples, or to reduce complexity where it isn't needed. If you have any questions about customizing these templates, start a support request, or make a post in the user forum.

Both of our existing samples are available for two workflows: with and without case management. Either way, they work best if your phone interviewers have a good connection to the internet, but this is not required. You can read more about how connectivity affects these workflows in part 2 of this guide.

4.1 The basic case management CATI starter kit

The basic CATI starter kit is the simplest possible CATI system that uses case management to track the progress of phone interviewers making contact with respondents. For simplicity, this sample form doesn't take many situations into account. For example, it assumes that a respondent has only one phone number, and if that phone number is wrong, that it won't be possible to interview the respondent.

4.2 The advanced case management CATI starter kit

The advanced CATI starter kit was designed to fill some of the gaps that the basic CATI system left out. To understand the advanced CATI starter kit, we recommend that you read about, and even test, the basic starter kit first to help you understand. The advanced sample tries to help you deal with a complex world in which respondents have many phone numbers, where some of those belong to other people, and where some of those phone numbers are better than others for reaching the respondent. The advanced sample form tries to help you deal with all these situations and more.

4.3 Non-case management starter kit (basic and advanced)

The non-case management starter kit includes both workflow samples described in 4.1. and 4.2. without the case management functionality. For that, it uses the table-list field plug-in to embed a pre-loaded table of case records into the form itself. Instead of opening a case from the Manage Cases menu in the SurveyCTO Collect app, the enumerator will open the form from Fill Blank Form, then pick the case from the table in the form. Read this part of the guide to help you make a choice between the case management and non-case management option.

5. Web forms as an option

This guide focuses on CATI on Android, but CATI can work well on web forms too, so long as your team has a good internet connection. Team members with laptops may prefer this.

However, automated dialing with the call-phone-number field plug-in, which works well on Android and iOS, may not be ideal for web form-driven CATI. If you click on the call button in a web form, it may open an application you cannot track (e.g. in Chrome, it'll suggest launching a call on an Android phone logged into the same Google account, and Firefox will ask which app to open).

However, you can also use HTML tags to initiate app-specific call completion. For example, if you wanted team members to make calls using Skype, the following HTML code would work:

 <a href="skype:${phone_number}?call">${phone_number}</a>

For the field with the above HTML, you can give that field a relevance expression of:

${deviceid} = '(web)'

And the field with the call-phone-number field plug-in can have a relevance expression of:

${deviceid} != '(web)'

That way, Skype will be used in web forms, and the field plug-in will be used on Android and iOS. The new device-info() function (read more here) can also tell the form about the system running your form.

6. Phone survey data quality

Phone survey data quality procedures are not necessarily that different in CATI survey interviewing. Many of the principles for collecting high quality data remain the same. In fact, the stakes for well-designed forms that take care of every possibility of a call not going to plan are even higher for survey success, as well as for data quality, given that interviews are taking place over the phone. One of the key differences is that because enumerators may be calling multiple times, you may have to review several form submissions when assessing a single interview.

SurveyCTO's audio audit feature does not work consistently with phone calls, due to limitations imposed by Google and Android hardware partners on phone call recording. Depending on the Android version installed on your device, audio audits might record a.) both sides of the conversation, b.) only the interviewer, or c.) neither. In brief, Android versions 4 through 7 allow for recording both sides of the conversation, and some success can be had with Android version 8. After version 8, call recording is not possible. Nevertheless, the early release version of SurveyCTO Collect (version 2.70.1) improves call recording on the Android versions that do allow call recording.

If recordings of phone calls are a non-negotiable requirement of your project, it is possible to achieve it through a field plug-in integration with a cloud telephony service such as Exotel, mentioned above. Another way to record calls is by interviewing respondents over speakerphone, but recordings of speakerphone will be less clear, and it will pose privacy and confidentiality concerns.

New: the early release version of SurveyCTO Collect supports the new phone-call-log() function. Using this function will provide you with more information about all the calls that were made while the form was open.

 7. Training your team remotely

Whether you need to prepare a field data collection team to work from home, or start from scratch with recruitment, you'll need to think on how to train your team remotely. At the time of writing this (April 2020), most countries in the world are in the midst of lockdowns and stay-at-home orders. As a result, much of how you used to operate with training isn't safe to do now. However, we have compiled a remote training guide for CATI to help you adapt. In summary, our suggestions are:

  • Slide presentations can still help. Think about recording a presentation to distribute, and remember to design slides for small screens.
  • Does your study have complex concepts? Consider developing videos to help make the hard parts easier to understand.
  • Design forms with instructional content. Fields can display text, images, audio for playback, and videos. Use these capabilities for remote training.
  • Prepare quizzes and comprehension tests as forms for trainees to complete, so you can be sure they understand the material.
  • Do practice phone interviews with team members. Consider preparing scripted sequences of actions (e.g. no answer, answer claiming to be the spouse of respondent) to compare against submitted data for practice interviews. If entirely scripted, you'll know what the answers and actions recorded by the interviewer should have been.

Do take a look at our full training guide as well. We'll be looking to improve this resource over time, and your feedback is always welcome.

8. Monitoring your project's progress

To help get you started, we developed an example Google Sheets CATI dashboard for you, to illustrate one approach to maintaining oversight of survey progress and adherence to protocol. Google Sheets is one of many tools you can use to make a dashboard, and is a good choice, given SurveyCTO's support for publishing to Google Sheets.

For more resources on dashboards, check out our blog posts on Google Sheet dashboards here and here, also our guide to integration with Microsoft Power BI.

9. Continuing your work

You might consider some other augmentations to your CATI system. For example, Zapier can be used to generate calendar events to help organize work, similar to what is discussed in this blog post.

Beyond the above, there is a lot of excellent advice available today about CATI, and adopting CATI in the midst of our current crisis. A select few resources have been collected in this article. However, we advise that you do further reading.

If you have further questions specifically about this resource, post them in the user forum. Paid users can submit a support ticket for fast, professional support on customizing this starter kit.

Please continue to part 2 of the guide and see the next part on the basic case management sample workflow for CATI.

Do you have thoughts on this guide? We'd love to hear them! Feel free to fill out this feedback form.

7 Comments

  • 0
    Avatar
    Amrik Cooper

    This guide has been improved since it's launch! See the new heading 7 for an example Google Sheet dashboard for tracking CATI.

    Do you have feedback on this guide? Write to support@surveycto.com.

     

  • 0
    Avatar
    Amrik Cooper

    This guide has new improvements! We've added version 2 of the accompanying sample form that uses a field plug-in for launching the phone dialer on Android devices.

    Field plug-ins are a feature of SurveyCTO 2.70, which is now available! Learn more in our latest blog post.

    Do you have feedback on this guide? Write to support@surveycto.com.

  • 0
    Avatar
    Amrik Cooper

    If you read our CATI starter kit and found the server dataset publishing side of things difficult to understand, we have a new resource for you! See the bottom of the "Overview" section of the guide for a link to a new webinar recording, all about server dataset publishing workflows.

  • 0
    Avatar
    Amrik Cooper

    A major update to the SurveyCTO computer-assisted telephone interviewing (CATI) starter kit has been launched! The starter kit is now a three-part guide:

    1. Part 1: Foundational advice on whether CATI is right for your project, what's required to adopt CATI, and how to prepare for CATI.
    2. Part 2: The basic CATI sample form workflow (previously released).
    3. Part 3: A BRAND NEW advanced CATI sample workflow!

    The advanced workflow builds on the basic sample workflow, and now deals with many more real-world situations. For example, respondents can have many phone numbers associated with them, and not all of the phone numbers will be owned personally by the respondent. One phone number may also be better than others. The advanced sample workflow deals with all of these problems.

    Those who have read the previous version of the guide will be familiar with some of part 1 and most of part 2. More improvements are coming soon. Stay tuned for future announcements!

  • 0
    Avatar
    Amrik Cooper

    The computer-assisted telephone interviewing (CATI) starter kit has been updated again. It now takes advantage of the very latest Android app features, available in our early release version of the app. We recommend adopting the early release for the best possible CATI experience.

  • 0
    Avatar
    Marta Costa

    We have a new resource to help CATI projects, all about remotely training your team to understand your project, and get them working from home. It features:

    1. slides to help users setting up SurveyCTO Collect on their devices, 
    2. a model quiz for assessing knowledge, and 
    3. a suggested approach on how to run mock interviews, where you'll assess competence using your CATI system.

    Check out this new article here: Training enumerators remotely for CATI.

  • 0
    Avatar
    Marta Costa

    Our CATI starter kit was expanded to include a non-case management version of the basic and advanced sample workflows!

    If you would like to learn more about the differences between a workflow with and without case management, take a look at this new article: How to use the CATI starter kit without Case Management.

    Additionally, if you're using the early release version of SurveyCTO Collect for Android, version 2.702 is available now, adding some great new security features.

Please sign in to leave a comment.