6

I'm experimenting with adding fields to attribute table data in QGIS 2.14.5LTR (mainly *.csv files) and stumble into a variety of issues or some things I do not understand, respectively. I imported a csv file via import text file as layer button, drag and and dropped the same file to the project and saved the latter as csv from QGIS.

My s(i/a)mple project looks like this (note the edit state of the layers):

enter image description here

1st scenario - import *.csv file and trying to add a field with field calculator.

When opening a *.csv file and trying to do so, I'm only able to add a virtual field to the table (Which can obviously not be used for joins, which is the aim of the game in the end). Strange enough: When dragging the same file into the project, I'm able to toggle editing mode and add new fields.

enter image description here

The imported *.csv file looks like this (no column headers):

92891A14.1;Eigenausbau;92891A14;09475169;Bayern;Hof, Landkreis;95131;Schwarzenbach;Döbra;Tannenweg;3;;2;4475738;5571977;272;11,99;11300;;30001;;100001;
92891A17.32;Eigenausbau;92891A17;09475169;Bayern;Hof, Landkreis;95131;Schwarzenbach;Schwarzenbach;Thiemitztalstr.;11;;1;4472917;5572009;432;16,61;12430;;30001;;90001;
92891A30.31;Eigenausbau;92891A30;09475169;Bayern;Hof, Landkreis;95131;Schwarzenbach;Schwarzenbach;Hochstr.;26;;15;4473440;5571918;147;4,78;13560;;30001;;100001;
92891A1.69;Eigenausbau;92891A1;09475169;Bayern;Hof, Landkreis;95131;Schwarzenbach;Schwarzenbach;Hauptstr.;16;;1;4473041;5571724;428;16,93;12430;;30001;;90001;

2nd scenario - saving the opened csv file as csv

Done so, I'm now able to switch to toggle editing mode and add a new field to the layer the usual way:

enter image description here

The saved *.csv file looks like

field_1,field_2,field_3,field_4,field_5,field_6,field_7,field_8,field_9,field_10,field_11,field_12,field_13,field_14,field_15,field_16,field_17,field_18,field_19,field_20,field_21,field_22
92891A17.32,Eigenausbau,92891A17,9475169,Bayern,"Hof, Landkreis",95131,Schwarzenbach,Schwarzenbach,Thiemitztalstr.,11,,1,4472917,5572009,432,16.61,12430,,30001,,90001
92891A30.31,Eigenausbau,92891A30,9475169,Bayern,"Hof, Landkreis",95131,Schwarzenbach,Schwarzenbach,Hochstr.,26,,15,4473440,5571918,147,4.78,13560,,30001,,100001
92891A1.69,Eigenausbau,92891A1,9475169,Bayern,"Hof, Landkreis",95131,Schwarzenbach,Schwarzenbach,Hauptstr.,16,,1,4473041,5571724,428,16.93,12430,,30001,,90001
92891A17.31,Eigenausbau,92891A17,9475169,Bayern,"Hof, Landkreis",95131,Schwarzenbach,Schwarzenbach,Thiemitztalstr.,22,,1,4472928,5572055,428,16.47,12430,,30001,,90001

Note: re-importing the saved *.csv layer leeds to the same issue as in 1st scenario

3rd scenario - Trying to add a field to the saved layer via field calculator from processing toolbox

Further more, attribute tables can not be selected as input layers for the field calculator from the processing toolbox (note the empty dropdown):

enter image description here

Questions:

  • What is the difference between the two *.csv layers in 1st and 2nd scenario (or the different ways they come into the project)? Why can the saved layer be edited and have fields added and the layer imported from a text file can not, but can, when dragged and dropped to the project?

  • What is the difference between all the field calculators and is there a way to add fields to a table with any processing tool?

  • Maybe a stupid question, but why don't you add the columns in excel before importing to QGIS if you're doing it manually in QGIS anyway? Then you could implement the join without having to manipulate your csv in QGIS. – jbalk Oct 19 '16 at 3:59
3

Interesting! I don't usually drag and drop files, but I could add fields to a CSV layer when I tried this. Which feels wrong :)

The changes weren't saved when I committed the changes. I could only get the changes to commit if I saved it as a new CSV file using save as... (Tested on QGIS 2.16.3)

I usually use CSV as a quick way of getting data from python scripts into QGIS, by writing WKT into a CSV. I have to use TAB or pipe (|) as a delimiter because WKT contains commas. If you drag-and-drop you don't get the option to set the delimiter and it assumes commas.

Take a look at the layer capabilities with the different scenarios (use Layer Properties > Metadata and scroll down the Properties section to look for Capabilities)

If you import a delimited layer (with geometry) you should see...

Create Spatial Index, Fast Access to Features at ID, Curved Geometries

If I drag in a CSV file, or import one without a geometry, I see

Add Features

Notice that in both cases, there isn't an "Add attributes" capability. This might explain why I need to save as a new CSV.

The GDAL/OGR CSV driver doesn't allow in-place edits (no 'Add features' or 'Add attributes' capabilities).

I suspect that when you drag in a CSV file without a geometry, QGIS treats it as a text file. If you import a csv and the geometry is recognised, the GDAL/OGR driver is used, and it won't let you make edits.

4

Trying to give an answer:

When adding a Delimited Text Layer (better switch to the english interface to have the correct terms at hand), you just create a connection to that file, not fully importing it. You can see that when you hover the mouse over the layer entry in the layers panel, showing a yellow pop-up after a while.

Completely imported layers like shp-files have their full paths shown there, with csv you see the path and kind of a query string like the ones you have when connecting to a database.

This was the standard way for adding delimited text data to QGIS. The drag-and drop way is fairly new and did not work for such data beforehand as I remember.

So the difference of the two scenarios might be:

  1. Add delimited data with coordinates (connection to that file only) and save afterwards to shp or whatever to be able to edit it.

  2. drag-and-drop any kind of tabular data, contained coordinates are just handled as attributes.

Leaving the quirks with csv aside, I think you should improve your workflow. CSV is definately not the format you would want to work with in a bigger scale. It has the charms of being human readable and is good for exports etc., but I think it's not very efficient and save doing more complex stuff with it, even if it is possible.

As you did not mention how those csv come to life, I expect that these are exports from other software. Check if there is a possibility to receive the data in another format. OpenOffice ODS-Files for example are easily added by drag-and-drop as well and editable afterwards.

3a. Why should you want to use the field calculator from the Toolbox and not just the one in the attribute table of your layer (Abacus-symbol)? There you can also add and delete fields. I think the field calculator in the Toolbox was recently added to be used with the modeller or to make changes to a layer and save the results to a new or temporary file (instead of changing the original)

3b. If you really need a processing tool to add fields besides of the field calculator, there is the "Refactor fields" function. But again, this will not make changes to your current layer, but "process" a new one.

  • 1
    Drag and drop has always worked. The main difference is the first one uses the QGIS provider for csv and the drag and drop uses OGR. Both do the same thing but a little different and don't both support the same features. – Nathan W Oct 18 '16 at 21:14
  • You can add fields because gdal/ogr 2.0 supports it. The QGIS provider currently doesn't – Nathan W Oct 18 '16 at 21:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.