Purging data on your server

Unless you delete collected data yourself, it will build up over time on a SurveyCTO server, as long as it has an active subscription (read more below). Even though SurveyCTO offers unlimited storage, it is not good practice to leave collected data on your server for too long, particularly if that data is sensitive, and especially if you’ve chosen not to encrypt your data. Old data can also slow down the server as it piles up over time.

The most basic measure you have against the build up of old data is to delete a form, deleting all of its data along with it. Alternatively, you can delete collected data up to a specified date, or delete specific form submissions. You can also schedule data to be purged automatically. All four options are described below.

Keep in mind, though, that in order to use SurveyCTO’s Data Explorer feature, the form and data must be present on the server. If the Data Explorer is important to what you’re doing, you might want to wait on deleting or purging data from the server. Alternatively, look into data visualization solutions outside SurveyCTO.

Important preparation

Download, save, and backup data from closed surveys or projects so that it can be accessed again in the future. This is a crucial first step, so this advice will be repeated throughout this article.

We advise frequent exports anyway, so you can start running preliminary analysis and data consistency checks along the way.

Option 1: Delete a form and its related data

This will delete the form, all of its data, and the history of all prior versions of the form. This is a good option if you don’t need your form any more, and you would like to delete all of the associated data in one step.

  1. Go to the Design tab.
  2. Under the title of the form you would like to delete, select Delete.
  3. Check the required boxes indicating that you already have the form definition and data (which you can also download from this screen by clicking on the download links).
  4. Select the red Delete button.

Preparing_to_delete_a_form.png

Option 2: Purge form data by date

This option will purge all submissions up to a date of your choosing (leaving the form itself intact, as well as the history of prior form versions). This will allow you to clear out older data that you have already backed up or downloaded elsewhere, while keeping data you still need on the server. For example, you might want to do this to delete pilot survey data that isn’t required.

  1. Go to the Monitor tab.
  2. Under Form submissions and dataset data, locate the form for which you would like to purge data.
  3. Under the form's title, select Purge form data.
  4. Select Purge by date.
  5. Select a date. All data from before this date is to be deleted, but data submitted on the date selected will be kept. Ensure this data has already been downloaded and backed up.
  6. If the review and corrections workflow is enabled for the form question, you’ll have the opportunity to include submissions by review status.
  7. Select Delete, and then confirm deletion.

Purging_by_date_2.png

Option 3: Purge individual submissions

This option will purge a specific list of submissions that you wish to permanently remove from a form. You might want to do this to clean your dataset of unwanted submissions when you’re certain that you don’t want to see these records in future exports.

  1. Find the unique KEY, or UUID, for each filled-out form you would like to purge from the server. Locate these in either the Data Explorer or in exported data.
  2. Go to the Monitor tab.
  3. Under Form submissions and dataset data, go to the form for which you would like to purge one or more specific form submissions.
  4. Under a form’s title, select Purge form data.
  5. Select Purge specific submissions.
  6. Enter the KEY for each submission you would like to delete, each separated by a comma.
  7. Select Delete, and confirm deletion.

Purging_by_key_2.png

Option 4: Setup auto-purge

If you would like, you can also configure the server to automatically purge data over a certain number of days old. More advanced users who use SurveyCTO as a data collection front-end while using one of SurveyCTO’s integrations to automatically published data into a downstream system, might want to limit build up of old data on their SurveyCTO server.

Important: Because data is going to be automatically deleted, be sure that data is already being downloaded and backed up.

  1. Go to the Configure tab.
  2. Under Server settings, select Storage.
  3. Check the box for Auto-purge submissions.
  4. In the box just below, enter the number of days submissions should be kept on the server (90 to delete submissions older than 90 days, 120 to delete submissions older than 120 days, etc).
  5. Be sure to set a number of days that gives you time to react, in case any automated export process stops working for some reason.
  6. Select Update settings.

auto_purge.png

Automatic data purging closed servers

Lastly, as stated above, data will build up on your SurveyCTO server until you delete it yourself, or until your subscription lapses. If you close your account, turn off auto-charge ahead of your next billing date, or stop completing community engagements, your server will close. There is a 10 day window that follows account closure inside which you could reactivate your subscription to recover data. After 10 days, your server will be purged of collected data for security reasons, leaving un-deleted form definitions and server datasets with no data.

SurveyCTO Desktop exports contain deleted data

SurveyCTO Desktop keeps a cache of all of the data it exports, so even if it is deleted from the server, it stays within Desktop. To learn more, and how to delete this data, check out this support article.

This article is a companion to our core documentation: Managing data.

0 Comments

Article is closed for comments.