How to design forms that accept incomplete interviews

This article accompanies this sample form on checkpoints and this sample form that uses a password to remove field’s requirement. Please read the following and test the sample forms. Either save a copy of the sample forms in your Google Drive, or download them as an Excel workbook. You can then upload it to your server and follow along.

SurveyCTO form designs work exactly according to your specifications. This is useful until you get to a question that's required, and there's no longer someone there to give the answer! This happens in all sorts of situations. For example:

• A respondent's consent changes, and they no longer wish to participate in a survey.
• The respondent runs out of available time, and you know it won't be feasible to finish the interview later.
• You're interviewing a respondent over the phone and their battery dies or the network cuts you off, and you can't continue the survey later.

There is no way around a required question, and only forms that meet all their requirements (including answers for required questions) can be finalized and submitted to the server. 

So what can you do?

You could save the form for later completion if you're working on SurveyCTO Collect on a mobile device or as a logged-in web form user. Save your progress in either scenario. In SurveyCTO Collect, press the back button, and select the option to Save Changes. As a web form user, click on the Save Form button in the menu bar. Whenever you want to continue filling-out a form, either on a mobile device or as a logged-in user in web collect, open Edit Saved Form and pick the partially completed form you'd like to resume.

Tip: Giving your forms an instance name will help you tell partially completed forms apart.

However, it may not be feasible to collect the rest of the data. What then?

If you want to recover the partial data you do have and send it to the server, there are a few strategies you can use. The first two strategies are proactive approaches that require thinking ahead, designing your form's relevance to accommodate this requirement. The last option is a reactive strategy that allows you to recover data, even if you hadn't thought about this in advance, but it works only in SurveyCTO Collect on a mobile device.

Option 1. Add relevance checkpoints to your form design

The best approach is to add checkpoints to your survey where you’ll ask whether the respondent is still available. All fields and groups of fields ahead will be in a group with a relevance condition based on the answer to that checkpoint field. For example, let's say the checkpoint field is a select_one field called "checkpoint" where "Yes" has a value of 1, and "No" has a value of 0 (pictured on the right). Next will be a group containing all remaining fields, and that group can have the relevance expression ${checkpoint}=1.

That way: If 'Yes', the survey will continue. If 'No', the survey will skip all further questions and reach the end. This is a discrete enough question that will cause no harm in the course of the interview, while it lets you finalize and send the submission to the Server. If your survey is divided into sections or modules, then you can add checkpoints in between sections. Otherwise, feel free to choose the best spots to include these checkpoints, taking into account your questionnaire. The number of checkpoints is up to you.

There is still a chance that the respondent becomes unavailable in between checkpoints. For these cases, ideally, you would have the option "don’t know", "refuse", or "no longer available" in your response option so that the enumerators select those until they reach the next checkpoint.

To add "don't know" and related choices to text, integer, and decimal type fields, check out our extra-buttons field plug-in.

See this sample form for a working example of this option. This is a good option to consider for any scenarios where you might lose the respondent at some point, like with phone interviews.

Option 2. Include a password that makes fields no longer required

While the required field parameter conventionally takes either "yes" or "no" values, it also accepts any expression that results in a Boolean value (true or false). This means that if you’re using the spreadsheet form definition, it accepts any expression that evaluates true (required) or false (not required). Taking this into account, one can create a text field that will serve as a password, a number only known by the relevant agents of your project (e.g. a field supervisor). Let’s say that the text field is named “password”. If you insert the expression ${password}!=123456 in the required parameter of a field, the field will be required unless someone enters the number 123456 for the field “password”. That way, while the field will usually be required, those who know the password can make it not required, and therefore skippable. It would be just like a ON/OFF button that can easily remove the requirement of your fields, enabling the submission of any form partially filled-out. See this sample form for a working example of this option.

Note: This option is available only when editing a form design in spreadsheet format. If you're not familiar with this process, read option 3 for guidance (first step).

Further, a word of warning about this approach is that it will reduce the absolute possible size of your form design because this is a computationally intensive strategy. Every field that you would like to make no longer required using this strategy would have an identical required expression based on the same password field. While using this approach with a form with few fields could be functional, implementing this in a form with hundreds or thousands of fields could be VERY heavy lifting for web forms or mobile devices. To learn more, check out our documentation on maximum form length.

Option 3. Reload submission with an updated form definition

There is a feature in SurveyCTO Collect, available for version 2.51 or higher, which will let you open/resume a partially completed submission using an updated form definition (this feature is discussed in the product documentation on updating an existing form design). By updating your form design so that all fields are non-required, all your incomplete forms can be finalized and submitted to the server. You can use this feature to get your partially filled out forms by following the steps below. This option requires that you work in the spreadsheet definition version of your form design:

Prepare the form on the server console

1. Obtain the latest, deployed version of your form design in spreadsheet format. If you are working in the online form designer, click on the Save menu button in the top right, then click on Download form definition from the menu that opens to generate the Excel or Google Sheet form definition. Otherwise, from the Design tab, under the form's title, click on Download, then Form files. Under Deployed form version, click on the .xlsx file link to download the latest version of your form in Excel format.
2. Open the spreadsheet form design and rename the required column on the survey sheet to disable it. You might name it something like "not_required" instead.
3. If working in Excel, save your Excel workbook (not required in Google Sheets).
4. Deploy your not-required form version:
• If you're working in the online form designer, click on Upload, and click on the first +Select file button, pick and upload the form definition.
• Otherwise, on the design tab, find the form you want to update and click on the Upload button beneath the title, click on the first +Select file button, pick and upload the form definition.

Reload the form on SurveyCTO Collect

Download the latest version of your form design in SurveyCTO Collect: Do this manually from the Get Blank Form menu, or From the Install Blank Form menu, if SurveyCTO Collect is configured to automatically download updates. In SurveyCTO Collect, go into Admin Settings and scroll to the section entitled USER CAN ACCESS FORM ENTRY ITEMS. Under this section you will see a setting named Reload with Updated Form. Enable this option. Go to the Edit Saved Form Menu and tap on any partially completed submissions. Once the form loads, tap on the 3-dot icon in the upper right corner and choose the option to Reload with Updated Form. Finalize and Submit the form.

Very importantly with option 3, if you still have data to collect, you will absolutely want to make sure no one is using the not-required version of your form! Fields being required is the most basic data quality promoting feature of digital forms, and without that, you'll have no idea which fields were skipped intentionally or not (and you probably want them all completed). So, if you want to continue data collection afterwards, you'll have to deploy a form update where you reverse this change so the fields are required again. You can simply reverse the change you made in step 2 and continue with the rest of the steps to update the form on the server and on devices.

Do you have thoughts on this support article? We'd love to hear them! Feel free to fill out this feedback form.