Offline GPS and maps using MBTiles

This is a companion article to the help topic Collecting GPS data, but that documentation is not a prerequisite. You should understand how to use the geopoint, geoshape, and geotrace field types. At the time of writing this, offline map support is available only for Android, not for iOS devices or web collect.

SurveyCTO is able to record GPS coordinates. Even if you are collecting data in an area without an internet connection, this is usually fine, since the geopoint field does not need access to the internet to work, simply relying on the device’s built-in GPS chip. However, if you need to select points on a map, such as with the geoshape and geotrace field types, or using the "placement-map" appearance, you will need to view the map in order to place the points. While connected to the internet, the map will download to your device as needed in order to place the GPS points. It will even cache that map so it can be used when you are next offline.

However, what if data will be collected offline, and you need a reliable way to make sure the map is always available? You can’t always guarantee the cache will keep as long as you need it, and it can take a while to open maps in Collect on each device. Luckily, SurveyCTO supports offline maps by using MBTiles. They can even be used to create a custom map that looks far different than the standard Google Maps display.

This article will provide guidance on how to create MBTiles files and use them on SurveyCTO Collect for Android, as well as examples for you to practice on your own.

The file type “MBTiles” is always spelled with an “s” at the end, even if you are just talking about one file.

About MBTiles

MBTiles is a format developed by Mapbox for storing raster and vector map data. It was an export format of their software TileMill. While Mapbox stopped developing TileMill in 2015 (source), it was taken up by a community of developers, and it can still be used to create MBTiles files. If you would like to learn more about TileMill, check out the official community website. We have some information on using TileMill in this PDF, but we will focus more on the HOT Export Tool.

SurveyCTO only supports raster-based MBTiles, not vector-based MBTiles.

Creating MBTiles files with the HOT Export Tool

The easiest way to create MBTiles files is with the HOT Export Tool. It was created and is hosted by the Humanitarian OpenStreetMap Team. It is an online application available at this link:

https://export.hotosm.org/

You will need to create and login with an OpenStreetMap account.

This tool is free and easy-to-use, but has a limit of 10,000,000 nodes (points used by the GPS system to track location (source)), 3,000,000 km² (1,864,113 mi²), and 5,000 tiles (the tool will let you know immediately if you try to exceed these limits). If you can work within these limits, this can be an excellent tool for you to create your own MBTiles files.

Number of nodes can vary greatly by location. For example, in Massachusetts, only about a third of the state can be highlighted before reaching the node limit. In South Asia, the square-kilometer limit is reached before the node limit.

In the middle, click on Start Exporting to get started, creating and logging into an account if necessary. Then, follow these steps:

On the right, get a view of the area you would like to make into an MBTiles file.
Use the tools on the right to select an area.
On the 1 Describe tab, give details about the map. Only the Name is required.
On the 2 Formats tab, checkmark MBTiles. You can leave the other formats unchecked.
On the 3 Data tab, for Source, select OpenStreetMap.
For Zoom Range, select the widest range you can that is compatible. A higher number, such as 20, means better resolution, but a smaller number, such as 0, allows for more zoom-out. Have the widest range that works best for you. A smaller map area allows for a greater zoom range.
On the 4 Summary tab, click Create Export.

None of the boxes on this tab need to be checked.

Wait for the MBTiles file to be ready, and then download the ZIP file containing the file. You will get an email when the file is ready, so you can leave and come back.
Unzip the file. You will need the .mbtiles file, but not the .geojson file.

Note: Special thanks to the Open Map Kit for mentioning this tool.

Other methods

While HOT Export Tool is a quick-and-easy way to create your own, free MBTiles files, there are many other ways to get the files you need. Use the method that works best for you! Here is a list of some sources to download or create MBTiles files, but this list is by no means comprehensive, so feel free to do your own research.

OpenMapTiles (pre-made)
TileMill (see this PDF for more info)
MapTiler
QGIS
Maperitive

Using MBTiles with SurveyCTO Collect for Android

Once you have your MBTiles file ready, you can move it to the layers folder, and use it in your form to help collect map data!

1. Moving the files to your Android device

There are several ways to get the MBTiles file onto your device:

Since this kind of map data rarely contains sensitive data (but sometimes it does, so be cautious), you can upload it to the cloud, and then download it to the device. (For example, in the "Practice" section of this article below, you’ll notice that a few of the practice MBTiles files are on Google Drive, and you can click on those links on your mobile device to download them directly.)
Distribute MBTiles files via downloadable links.
If you create the MBTiles file on Windows, you can plug in your Android device with a USB cable, and simply drag-and-drop the file to the correct folder within the layers folder (see the next section for more details).
On MacOS, use Android File Transfer to copy the MBTiles files to Android.

2. Installing your MBTiles files

Once the MBTiles file are on your device, follow these steps to install them in SurveyCTO Collect. Repeat these steps for each MBTiles file you would like to install.

On your Android device, navigate to this folder:
```
/Android/data/com.surveycto.collect.android/files/config/layers/
```
We will call this the layers folder.
Create a folder for the MBTiles file. The name of the folder will be the name of the layer shown to the enumerator.
Move or copy the MBTiles file to its folder within the layers folder. Make sure there is only one file per folder in the layers folder.

Here in the layers folder, there are two folders, each with a single MBTiles file. With this setup, there will be two offline map layers to choose from: "Cambridge" and "World".

These instructions are up-to-date as of SurveyCTO Collect 2.70.6. Older versions of SurveyCTO Collect had a different layers folder location. When you update to 2.70.6, you will need to move the folders in the older /SurveyCTO/layers/ location to the new location mentioned above.

If you want to test multiple files in the same layers folder for yourself, download this ZIP file to your device, unzip it, and move the created folder to the layers folder. You will find that when trying to use it for a layer, none of those maps will load.

3. Using MBTiles in a form

As per usual, when you get to a field that uses a GPS map, tap the View or Change Location button to open the map.
When prompted, zoom to the current location, if applicable.
On the top-right, tap the layers button (the two layered squares, third symbol from the bottom).
Select the layer you would like to use. It will have the same name as its folder.

Troubleshooting

If your map is not appearing, try zooming out, then slowly zoom into the current location. The map will appear once you are zoomed to a size supported by the MBTiles file.

If you originally downloaded the file as a ZIP file, make sure it has been extracted, and move the MBTiles file to its folder in the layers folder, not the ZIP file.

If you have any other issues, you can reach out to SurveyCTO support. We are always happy to help! If you do not have a paid subscription, you can post a question in the community forums. Note that advanced methods for producing MBTiles files are a little outside the scope of our support. We will at least be able to assist confirming a.) whether you're working with a raster-based file, and b.) whether you've followed instructions on how to use MBTiles with SurveyCTO Collect.

Practice

Here are a few examples to practice using MBTiles in SurveyCTO. Following these steps can be great for practicing the steps mentioned above. Have a deployed form ready to test with. You can either create your own, or use the one here.

Example 1: World map

This is an example of how a new map looks over the standard map. It will also show you what happens when you zoom in too much. This map is fairly low resolution, so it will not take long to disappear.

Create a new folder in the layers folder named “World Countries”.
On your mobile device, download the zipped world map MBTiles file at this link.
Unzip the file.
Move the file that is extracted to the folder /layers/World Countries/ you had created.
Open a form that includes a field for recording GPS coordinates on a map (like a geopoint field with the "placement-map" appearance).
When going to choose a point, zoom all the way out.
On the right, tap the layers button.
Select World Countries. The world map will change from the default Google Maps view to a colorful map of world countries.
Slowly zoom into a country. Eventually, the colors will be replaced by the standard Google Map.

Example 2: Finding Cambridge

In this example, you will find and zoom into a city in Massachusetts. The map was created in HOT Export Tool, and has a zoom range of about 0-18, so it should give you a good idea of what your own zoom range should be.

Create a new folder in the layers folder named “Cambridge”.
Download this zipped file to your device.
Unzip the file, and move it to the folder created in step 1.
Turn off the device’s WiFi and mobile data (if applicable).
Clear the cache from your SurveyCTO Collect app (this is to get rid of any cached map info, so you can better simulate the conditions enumerators may work in). To do so, go to the app’s info, then storage settings, then clear the cache. The exact settings may be different for each device. Do NOT clear any other data, just the cache.
Open a form that has you choose GPS coordinates.
On the right, tap the layers button, and select Cambridge.
On the “map”, which will mostly just be a series of small squares, zoom all the way out, and go to the very top of the “map”. Then, scroll left or right until you see a patch of map. It will be of the northeast United States and eastern Canada.
On this area, slowly zoom in. As you zoom in, the colored patch will cover less area, so be sure to keep moving to the viewable portion so it is always in the middle.
Keep zooming in like this until the map disappears. Then, slowly zoom out so it appears again.
Try recording a GPS point from this map.
Turn WiFi and mobile data back on. As Google Maps downloads, you’ll find that the MBTiles layer is on top of the Google Map.

Example 3: Creating your own MBTiles file

In this example, you will create your own MBTiles file, and zoom in on your own location.

Go to https://export.hotosm.org/, create an account if necessary, and login.
On the map on the right, zoom into where you are now. Zoom in enough so the entire city where you live is visible.
On the right, click the red Box button.
Click once in the corner of the area you would like to select.
Move the mouse until the entire city where you are now is selected, and then click again. An area should now be highlighted in blue.
Follow the steps above under Creating MBTiles files with the HOT Export Tool to complete creating an MBTiles file.
You will get an email with a link to the zipped MBTiles file. Download that ZIP file to your device.
Create a folder for the MBTiles file in the layers folder.
Unzip the file, and move it to the folder created in step 8.
Optional: Clear the cache of the SurveyCTO Collect app, and turn off WiFi and mobile data. This will simulate situations where enumerators are offline.
Open a SurveyCTO form that includes a field for recording GPS coordinates.
When you get to the map, go to your current location, which will be marked by a red marker.
Select the layer you had created and added.
Try zooming in and out. Depending on your zoom range, the map you created will appear and disappear at certain zoom points.
Try picking coordinates on this map.