QGIS 3.2 - Forcing column type when importing csv

Question

I've got this csv file that I'm trying to import to QGIS 3.2. It's just a data table with no geometry attached.

When I just drag it from the Browser on the left-hand side, all the columns get incorrectly read in as string. However, when I use the "Open Data Source Manager" and import the file through there, the column types are all correct (first one is integer and the others are double). But I really didn't specify that the columns should be double instead of string.

So, finally, my question is: how/where can I try to force certain column type at import for each of my columns?

Also, extra brownie points to whoever knows what's going on in the background that makes dragging a CSV from the Browser different from using the "Open Data Source Manager".

Answer 1

QGIS uses OGR in the background, and OGR interprets all columns as strings.

The OGR CSV driver returns all attribute columns as string data types if no field type information file (with .csvt extension) is available.

Using CSVT file with the same name as the CSV file allows you to specify types of columns in a CSV file.

Limited type recognition can be done for Integer, Real, String, Date (YYYY-MM-DD), Time (HH:MM:SS+nn), DateTime (YYYY-MM-DD HH:MM:SS+nn) columns through a descriptive file with the same name as the CSV file, but a .csvt extension. In a single line the types for each column have to be listed with double quotes and be comma separated (e.g., "Integer","String"). It is also possible to specify explicitly the width and precision of each column, e.g. "Integer(5)","Real(10.7)","String(15)".

The difference between dragging a CSV from the Browser and using the "Open Data Source Manager" is that "Open Data Source Manager (ODSM)" guesses data types of columns.

Sample .csv and .csvt files:

sample.csv

1,high,2.3
2,low,5
3,low,7.2

sample.csvt

"Integer(5)","String(10)","Real(3.5)"

If you import the file to QGIS using Browser or ODSM, the attribute table looks like this:

Further information, please look at Comma Separated Value (.csv)

Answer 2

Another work-around is to specify the field types while in Excel, and then save as an xlsx file. When you import the .xlsx file to QGIS, your field types will be maintained. This does not work if you specify the field types in Excel and then save as CSV.

Answer 3

I'm going to suggest more of a theoretical/best practice answer rather than a technical solution.

Whilst QGIS and other applications are fantastic at being flexible enough to allow us to visualize simple datasets (like a CSV file), these practices are not what I would consider 'best practice', particular for longer term, sustainable, reusable solutions.

The CSV file contains no geometry, so im assuming you will be joining your data to another table that does have geometry. ie: You will be performing a join. This makes the data types more important. The assumption also being that it will 'exist' without any security/access permissions, so there is risk there as well. (please do excuse my enterprise information management principle ego - im working on it ok!)

In your instance, with your question being:

how/where can I try to force certain column type at import for each of my columns?

My humble answer is - Import your data into a more appropriate data structure that allows for better execution of the capability you require.

A simple database in this instance would be a more appropriate data structure (geopackage for instance). This would allow data types to be set, joins to be created inside a self-managed container, set precision on fields, date fields, simple security mechanisms, performance benefits and most importantly, data consistency for future use.