SurveyCTO supports Stata data analysis software through providing users with a .do file script for Stata, aimed at helping Stata users get started with data cleaning and analysis a bit faster. SurveyCTO’s Stata .do file template will import, format, label and clean your data. Click on the link above for details. There is some scope for SurveyCTO’s .do file template to not work perfectly out of the box due to some of Stata’s requirements and the .do file templates assumptions, or because of certain form design choices and in some cases, some issues to do with how data was exported. Read on for the most common .do file template troubleshooting solutions.
Before we proceed with this article, we need to put forward a minor disclaimer: this guide is aimed at helping SurveyCTO users get the .do file as generated by SurveyCTO working with their raw data, also exported by SurveyCTO. We do not support Stata beyond this. Stata is a product of StataCorp who support their own product. StataCorp offer free technical support to registered owners of Stata. The answers to Stata-related questions might also be readily available online on Statalist, the official Stata user forum, or elsewhere.
Customize the working directory
The .do file template generated by SurveyCTO will work when placed on the root of the Stata working directory, along with the CSV file/s in the same location. If your files aren't located on the root of the working directory, you'll get an error saying that Stata can't find the files. Not all Stata users prefer to work like this, so if you have different directories for keeping CSV files and .do files, maybe different from the main directory, you’ll need to edit the .do file template. See the “initialize working directory (TO BE CUSTOMIZED)” heading in the template.
Try starting from a new Stata dataset
Firstly, to maybe save you some reading, if you have run your SurveyCTO .do file before and it worked but now you see missing values in Stata, it could be worth regenerating your Stata .dta file from scratch. SurveyCTO’s .do file template will append new rows of data to an existing .dta that it can find and failing that, it will create a new .dta file from your raw CSV data. Regenerating your .dta file can resolve problems like data type discrepancies in Stata which sometimes result in missing data, for example. Discrepancies might be caused by updates you make to your form while collecting data (see below). So either move your .dta file to a new location in your file system (if you want to keep it) or delete it and run your .do file again.
Check your data in Stata. Is data still missing? If yes, don’t worry, continue reading.
The problem might be in your data, not with Stata
If data is missing in Stata after importing it using SurveyCTO’s .do file, the problem might be either that data is missing from the data that is being imported into Stata in the first place (so it was never there to begin with), or one of the steps in the .do file resulted in values going missing. The first thing to check would be whether the values are there in the raw CSV data in the first place. Viewing raw CSV data is best done using software that won’t transform or format it. Our recommendation would be a text editor (like Notepad) but you could also use Excel or Google Sheets (note that Excel and Google Sheets can still automatically format values). While data isn’t displayed in easily readable columns when viewed as text, it should be possible to see whether the values that are missing in Stata are also missing in the raw CSV data. If data is missing in the raw CSV data, it will be worthwhile to consult this Reference Material article: Finding Missing Data. This article will guide users on how to locate missing data.
Another resource that might be useful if data is missing from certain fields in raw CSV data is this article: Missing data from old form versions. If you have updated your form design during the course of data collection, there are certain changes that might result in this but don’t worry as your data will be recoverable. For more details, see this help topic: Updating an existing form.
However, it might be that the data is present in your CSV file and missing in Stata after running the .do file. That will take some troubleshooting.
Date format conflicts between data and .do files
It might be that only date and time values like SubmissionDate, starttime and endtime are missing after import into Stata. The reason for this can vary and it will be valuable to go over some of the details of how SurveyCTO’s .do file is generated and how that might result in this issue. The server console generates the format of dates in exported CSV files and the .do file template’s date commands according to your computer's local language and locale settings. Each locale and language can have a different date format (e.g. with US English locale settings on a computer, dates will be in Month-Day-Year format).
Such problems more frequently occur when working in teams, where one person exports data and a .do file using their computer’s settings and someone else downloads new data from a computer with different settings. The result in Stata, when it does not recognise the format of a date value (or when a non-date value is passed), is that such values appear to go missing.
The best solution most often is to export data and a .do file on one computer. You might also discuss unifying computer settings with your colleagues. Alternatively, the .do file can be manually edited so that the date commands match the format of dates in the raw data.
General troubleshooting
If you’re struggling to work out why Stata seems to discard data or behave oddly when using our .do file template, beyond the above, our advice is the following.
- As above, first confirm the data is there in the CSV file to start with.
- If there are errors, we suggest that you Google them. StataList is the main Stata forum on the internet and has a wealth of advice, dealing with almost every Stata problem that’s happened before.
- You also might get a more detailed understanding of what’s happening by generating and examining a Stata log file.
- .do files in Stata are a list of commands to be run in a certain order. The whole list can be tripped up by a problem with just one of those commands. If the .do file itself doesn’t end in an error, there still might be a problem which you might tease out by running a shorter section of the .do file to narrow down where the problem is.
- Stata .do files can have sections of text declared as comments that Stata will ignore (“‘commenting out” that text). This can also be useful for troubleshooting .do files. Read more here.
We hope this helps! If you’re a user on a paid account, you’re welcome to write to the support team for more help. Go to the Support Center and click on the Submit support request link to start a support ticket. Users on the free community edition plan can submit public questions to the community on the user forum.
Do you have thoughts on this support article? We'd love to hear them! Feel free to fill out this feedback form.
0 Comments