This article is accompanied by several sample forms that you can find in this folder. Download the files, or make a copy to your own Google Drive.
Field supervisors perform a crucial role in surveys. They manage logistics, and they're the frontline of data quality, sitting in on face-to-face interviews when required, providing direct feedback to enumerators. In our current circumstances of May 2020, many survey projects have migrated to alternative data collection approaches like computer-assisted telephone interviewing (CATI), but what about frontline interview supervision? How do you continue to work to ensure data quality on the frontline?
In this article, we explore the idea of remote accompaniment over the phone for the purpose of supervision, where field supervisors can't be there in-person. In brief, remote accompaniment in CATI surveys is possible through conference calls used to dial a supervisor into the interview. Supervisors can sit in over several calls, and/or be connected to calls randomly, to provide oversight. In contrast, where in-person interviews and data collection are unavoidable, remote accompaniment can also take place over a standard phone call.
Read on for more details on this approach to remote accompaniment.
| Remote accompaniment is one of many quality control measures you might employ. Learn more about data quality control for phone surveys in this guide. |
Face-to-face interviews
While face-to-face interviews won't be possible for many projects at the time of writing this, in-person data collection will be unavoidable in some contexts (when providing essential services which have a data collection component). Nothing special is required to remotely accompany such work though. For years, the multitasking capabilities of smartphones have facilitated being on a call while performing other tasks (such as capturing data). Depending on your requirements, video calls could also help place a supervisor on location, bandwidth providing. However, nothing special is necessary - phone calls between two feature phones can facilitate remote accompaniment for the same price as a standard phone call.
It was previously possible to initiate a phone call by opening the phone app, but it is now also possible to launch a call directly from SurveyCTO Collect app, using the phone-call field plug-in. The form's design can also help make the decision of who to call for remote accompaniment, based on a schedule. One could also introduce randomization, so that some interviews are remotely accompanied and others are not.
We suggest that enumerators put the call on speakerphone, and that as a rule, supervisors mute their microphones, activating them only in exceptional circumstances; this will provide the most optimal circumstances for remote accompaniment. If an audit is planned that involves no interviews, enumerators might use earphones with an in-line microphone to discreetly stay in contact with the supervisor, discussing what they had observed.
To help you get started with ideas on how to handle this, see our sample form:
Sample form - Remote accompaniment for face-to-face interviews
| Tip: Think about designing a form supervisors can use to record their observations. This applies to any remote accompaniment scenario. |
Remote accompaniment for CATI surveys
The basics
By far the simplest way to set up a remote accompaniment with CATI interviews is to have a supervisor be available for a dedicated period of time to sit in on contact attempts and interviews. For most projects, the most important time to do this will be right at the start, during the pilot or the first week of interviews, so that the rest of fieldwork benefits from the direction and guidance supervisors can offer. Some time might be spent waiting while enumerators dial respondents who aren't picking up, but that's part of phone surveying. Also, this is likely smoother than contacting the respondent first, then adding the supervisor, since then the respondent will not have to wait for the enumerator to contact the supervisor.
The enumerator would first call their supervisor, and have them sit in the background of interview attempts. This allows for coaching of enumerators before calls. Each respondent would be dialed in on a new call, which will place the supervisor on hold initially, until the enumerator merges the two calls. Conference calling is commonly supported by most networks and devices (see this how-to guide or any other guide on how to initiate conference calling). We suggest that you confirm whether this is possible by doing some research, which may even inform your choice of mobile network. Also have enumerators practice setting up conference calls before they start calling respondents.
Before calling the respondent, the enumerator might share the respondent's unique study ID with the supervisor, so the supervisor can follow along based on what's known about the respondent, should they answer (in a remote supervision form, for example).
Here are two crucial points on the process:
- Supervision of calls should be something that's discussed as part of informed consent.
- For the clearest call possible, supervisors should mute themselves, intervening only if strictly required, or via instant messaging.
To help you get started with these ideas, take a look at this sample form:
Sample form - Basic remote accompaniment of CATI interview
Randomized accompaniment
Remote accompaniment throughout a survey project is often helpful, and supervisors can't be everywhere. However, leveraging randomization can provide you with good coverage. This can be an alternative or complement to the basic approach above. In contrast to the ad hoc accompaniment that's possible with traditional face-to-face interviews, we suggest that rules and procedures be designed into your form; that way, supervised interviews are distributed among different enumerators as evenly as possible.
With a pre-existing list of respondents, you have the option of randomizing the flagging of respondent records for supervision outside SurveyCTO. Alternatively, you can randomize inside the form design itself using the once(random()) expression. Keep the following in mind:
- The availability of supervisors, given the supervisor-to-enumerator ratio.
- The probability of reaching the respondent in any one call.
Given these factors, it won't always be possible to have a supervisor present with every interview you have flagged for remote accompaniment, so think about flagging a greater number than the minimum you require. Once you have decided on the probability, make form elements relevant in the design for initiating remote accompaniment. For example, if a random number is stored in a field called "rand", and 20% of interviews should have remote accompaniment, remote accompaniment fields can have this constraint:
${rand} < 0.2
There are different strategies you can use to choose the supervisor for the remote accompaniment. For example:
- Assign a specific supervisor to a specific enumerator beforehand.
- Randomly draw a number using the
once(random())function, and use that to select the supervisor. - Store a list of independently maintained supervisor contact details in a server dataset, allowing enumerators to pick one from a pre-loaded choice list and attempt to call them.
- The above list could be filtered based on the day of the week, based on a roster of availability (expression
format-date-time(today(), '%a')will return the day).
To explore the idea of adding randomization into this process, please see this sample form where the last two more recommendable options are exemplified:
Sample form - Advanced randomized remote accompaniment of CATI interview
| Tip: It might be in practice that the supervisor is less frequently available for certain enumerator's interviews than with others, resulting in less remote accompaniment for some enumerators. To adjust for this, and to provide yourself with the ability to increase the chance that an interview is flagged for remote accompaniment for individual enumerators, you can make the probability vary based on the identity of the enumerator. For example, probabilities could be stored in an attached server dataset, pre-loaded into the form based on the unique username value. This would allow you to easily modify the probability of requiring remote accompaniment for any one team member. |
Phone interviews using web forms
The SurveyCTO CATI starter kit is a cross-platform solution, partly thanks to the fact that phone interviewers can use web forms too. A desktop computer can then be used for data capture, which can also launch a call via a VOIP service like Skype (also an option for mobile devices). As you're probably aware, Skype supports multiple people on a call, so a group call can facilitate remote accompaniment. Enumerators can start a call with a supervisor, then inside Skype, add the respondent to the call. Skype can connect you for free if you have the respondent's Skype ID, but is also capable of connecting to cell phones and landlines if you provide enumerators with Skype credit.
As above, the supervisor should mute themselves on the call, and the presence of the supervisor should be addressed as part of informed consent.
See heading 5 in part 1 of the CATI starter kit for example Skype HTML code.
Alternatives and complements
Remember that listening in after an interview has finished is an option too. SurveyCTO Collect for Android version 2.70.6+ has improved support for the audio audit feature, allowing for call recording in Android versions 4.4 through 7 (later versions of Android disallow this).
However, for an ideal enterprise-grade call-recording solution, you might follow in the footsteps of J-PAL South Asia, who integrated with an Indian VOIP service, Exotel, using a field plug-in. Take a look at their field plug-in. Market equivalents to Exotel serving the country or countries you're looking to survey probably exist.
Solutions like Exotel can provide call recordings, call masking, improved call rates, and discounts for nonprofits.
Do you have thoughts on this guide? We'd love to hear them! Feel free to fill out this feedback form.
0 Comments