Using sensor meta-data

In SurveyCTO 2.50, we introduced new, experimental sensor meta-data field types. If you want to collect better and more accurate data, you can use device sensor data to get valuable insights about whether data is being collected according to plan. To supplement the product documentation on this feature, this article will share some perspective and discuss how to make the best use of sensor meta-data, as well as getting into some limitations. Toward the end, using sensor meta-data in combination with SurveyCTO's other quality control features is discussed.  

What do the new field types do?

Android devices can come with a number of sensors beyond GPS including an accelerometer, gyroscope, light sensor, microphone, among others. The new field types use these sensors to capture data during the survey that can provide users with an idea of:

• The light conditions around the device.
• How much the device moved.
• How loud the sounds were around the device.
• The pitch of the sounds around the device.
• An estimate of whether a conversation was taking place around the device.

Why is it useful to know the above information?

• Different types of data collection projects will involve different light conditions. For example, a household interview probably takes place inside the home of the respondent, in light conditions that may vary – but the intensity of that light will still be lower than the intensity of full sunlight outdoors. In general, if an observation should take place mostly indoors, outliers for light levels might be worth checking on.
• Most surveys involve an enumerator sitting, reading questions off a tablet screen, while typing answers.The enumerator might show the screen to the respondent from time to time, but on the whole the tablet shouldn’t move a whole lot. A high level of movement might suggest the device was being held while walking, or while driving in a vehicle, which may not have been according to plan. In contrast, in a form where enumerators are expected to walk around a school, clinic or farm, completing a series of observations on a checklist, you might expect plenty of movement.
• The loudness of sounds around the device during a survey  can tell you something about the conditions under which data was generated. Certainly, human voices fall inside a range in terms of volume. Is it expected that forms would be completed in loud places, with lots of background noise? Or in the context of your project, would that be more of an outlier? In either case, this is something you can get an indication of using device sensor data.
• The most common use case for SurveyCTO forms is to collect responses from a respondent during a face-to-face interview. This involves a conversation: the enumerator reads out questions and the respondent answers. Hence we’ve tried to measure the percentage of time during a survey that a conversation seems to be happening using the volume and pitch of the sounds around an Android device as indicators. More than other statistics and stream data, the conversation sensor data should be regarded with some skepticism, as it is possible that the current version of this feature will falsely identify non-human voice sounds as conversation in cases. As explained in the documentation, these are experimental features, so should be used with that in mind.

While each type of sensor data can be useful individually, they are even more powerful when used in combination. For example, in the case of a household study, one might expect mostly low light readings reflecting being indoors, nominal indications of movement, moderate sound levels over which a conversation would be taking place for the majority of the duration that the form is open.

What does that look like in a form, in practice?

You might use the following sensor_statistic fields with these parameters:

• sensor_statistic pct_light_level_between, with the appearance “min=5;max=500”, to get the percentage of time spent inside the lux range that likely reflects being indoors.
• sensor_statistic pct_sound_level_between, with the appearance “min=0;max=60”, for an indication of the amount of time where there was mostly a quiet space where a conversation might be heard, or with the appearance, “min=80”, for the amount of time that loud sounds were detectable, during which you might have doubts whether the respondent’s answers could be heard properly and recorded accurately.
• sensor_statistic pct_movement_between, with the appearance, “min=25;max=65”, for a rough sense of the duration of moderate to low movement, for a reflection of the device being held by hands. Equally, one might try using the sensor_statistic mean_movement field to see how it compares.
• sensor_statistic pct_conversation, to see what percentage of time a conversation is taking place. As in the product documentation, this is an experimental feature, so it may not always work well.

The above is by no means a firm recommendation on specifically how to use sensor statistics. Rather, this is a suggestion on where to start, if you happen to be doing household interviews.

Important caveats

Because a lot of natural variation is possible in the sensor statistics - different devices have sensors of different sensitivities, the ranges of light/movement/sound conditions that are normal for one project might be abnormal for another project - if you are interested in sensor statistics, you need to first establish a baseline. Add these fields into your forms, and observe some initial surveys. Compare the sensor data gathered to the conditions you observed, to get a baseline of what light, movement and sound statistics are recorded in some normal surveys.  This will give you a sense of what thresholds to use for your sensor statistics and what value ranges should be flagged as outliers.

In general, you also have a choice as to whether to flag desirable or undesirable ranges of values. Also consider adding a few questions for the enumerator at the end of your form designs to help gauge their sense of how long they were indoors or outdoors, or for how long a conversation was taking place while the form was open, to give additional context to any readings you get.

Given the experimental nature of these fields, aside from taking this data with a pinch of salt, we also ask that you include related sensor_stream fields with the default time period (by leaving the field’s appearance blank) along with their related sensor_statistics fields, even if you don’t have any specific plans to do analysis on the sensor stream data. It can still be useful for you to review a few sensor_stream files as a check against the sensor statistics being reported. Stream data can also be helpful for our team to review when you are giving us advice on how to improve these features. So, at least for the first project where you experiment with sensor data, add the relevant sensor_stream fields that inform the sensor_statistic fields you’re using. For example, if you’re using the sensor_statistic pct_conversation field, you should include the sensor_stream conversation field as well as the sensor_stream sound_level and sensor_stream sound_pitch fields in your form.

Combining sensor data with other features to create a cohesive data quality and review system

For concrete steps on how to combine sensor data with other quality promoting features, read this guide. 

You do not only have to use outlier checks with sensor statistics as in the guide though. For example, a popular check amongst our users is to flag surveys that have very short durations compared to the sample as a whole. Now, you can do the same with sensor statistic data. Using the household interview example from above, let’s say that during a pilot survey mean light readings fell between 50 and 250 lux. So you might decide to flag submissions for assessment in the review workflow where the lux value is greater than 500, just to allow for some variation (keeping in mind that an overcast day is about 1000 lux and full sunlight can be 10,000+ lux). As in the above guide, you could create a quality check that automatically flags submissions with a lux light level above 500 for review.

The future

Over time, we aim to use machine learning technology to automate even more of the quality control review that users do, and sensor data is critical to that effort (though it still some time away). If you’re excited about sensor data and have experiences to share, we would love to hear from you, so do get in touch! You can help make sensor meta-data as useful as possible for all SurveyCTO users.

Do you have thoughts on this support article? We'd love to hear them! Feel free to fill out this feedback form.