Understanding QGIS errors when importing .csv file?

Question

I am trying to import a .csv file as a layer in QGIS and keep getting a number of errors that I don't understand. Is there any documentation of QGIS errors available? Or can you point me towards a description of what QGIS sees as a valid .csv file? To me, the file looks ok in different editors.

The errors I get are (translated into English, so maybe not exact):

x data sets discarded because of invalid format

y data sets discarded because of invalid geometric definitions

the following lines were not loaded into QGIS because of errors:

invalid sentence format in line xyz, invalid x or y fields in line yxz, ...

there are zyx additional errors in the file.

I tried to find help on the internet, but I did not succeed. Now I don't know where to start because there are so many errors.

The problem exists in QGIS 2.0 (Dufour) and 2.8 (Wien). The errors look the same to me in both. I am trying to insert a .csv file that contains text, a time stamp, and lat and lon information.

The file header looks like this:

ID§$%text§$%timestamp§$%latitude§$%longitude 

As you can see, I chose a set of delimiters that are - hopefully - not present in any Tweet ;) At least for the few example lines QGIS shows when importing the file, delimiting the field worked.

The texts contain emojis that are shown as, e.g., in my bash terminal with locale "de_DE.UTF-8". So maybe they are not proper UTF8?

Answer 1

I found the solution for my problem. There were (probably) two "errors" in my .csv.

1. QGIS seems to detect that my data field called "text" contained exactly this - a text. Thus, it encoded it as a String (or character) data type. Within such a data field, however, double quotation marks are only allowed at beginning and end or not at all. Since my Tweets contained quotes within them, this seemed to cause the problem. Single quotation marks (the apostrophe) are no a problem (so I just converted them and since then it works). On the other hand, it does not make a difference whether the whole field is contained within quotation marks or not.

2. QGIS treats a sequence of characters that you enters as "custom delimiters" like a logical or, not and. Thus, when I used "$%&" as delimiters, it also split the record at a single "&". So my choice of delimiters also caused a problem. But I got the idea to use and alt-key character (pressing alt-key and a letter at the same time) as a delimiter from Branco's answer here. I used alt-key plus s and QGIS import the file without any problems.

Also, for the record and in case other German speaking users run into the same error: "Ungültiges Satzformat" translates into "invalid record format" which gets you on the right track while searching for a solution to this error. I ultimately got the idea that the problem has something to do with strings and quotation marks from the comments to this question.

Answer 2

What editor are you using to save the CSV?

I find that saving the CSV data through LibreOffice, rather than Microsoft Excel helps resolve this in certain cases, particularly on Mac OSX.