2

I try to load a Lidar point cloud in .xyz format to Grass (in QGIS 2.0.1).

I tried to input the following options to the 'separator' field: tab, 'tab', '\t'. Just for the unlikely case that it worked I tried alse ' ', 'space', 'newline', 'comma' and ';'.

Two things have changed. First: empty space between two apostrophes didn't work yesterday, but it works now (I don't know why). Phrasing it as 'space' still doesn't work.

Second: I found the data are actually separated by two or three spaces, not by a tab. When I set space as the separator, there is not the old wrong separator error, but Y coordinate seems wrong: it looks for it between the two spaces separating the fields. Writing ' ' (double space) to the separator field doesn't help. So I rephrased this question to fit my problem better: how to handle multiple space separator in Grass?

First few lines of the .xyz file look like this:

  3453939.450  5633900.030   361.670
  3453941.750  5633900.020   361.720

Other options are the default ones, just the "statistic" is "median" instead of "n".

2
  • Do you have the option to use a text editor and replace all double spaces with a single space? That would probably be the easiest. BTW, if you want to use the words for the separator paramter, then it's without quotes. as separator=space or sparator=comma. The equivalent is separator=' ' or separtor=','.
    – Micha
    Feb 28, 2014 at 10:10
  • Yes and no - the data I have are too big for ordinary editor (hundreds mega), but Vim can handle them. Well, can handle - it took half an hour to replace first two (double space) columns, and then I left for more than two hours and found Vim still working on the third column of the half-giga file. I must have done something wrong (running too much work at once, causing frequent read from/write to disc), but prospects of preprocessing much longer than running the Grass tool make me try to find some faster workaround.
    – Pavel V.
    Feb 28, 2014 at 13:05

2 Answers 2

2

As pointed out, Vim seems to be a wonderful text editor to handle large text files.

Here is what I did, after reading this tutorial for beginners and this question on StackOverflow.

:%s/ \+/ /g

The above command replaces multiple white spaces for a single whitespace (one can choose a different delimiter, though). It uses the following syntax:

:%s/text/replacement text/g search through the entire document for text and replace it with replacement text.

the \+ part addresses multiple whitespaces in the text argument.

I am not sure how long it takes to replace all the multiple spaces into your .xyz lidar files, but with the sample you provided the code worked. See below:

enter image description here

The result:

enter image description here


If the lidar text files are not that big, a simpler option could be a common text editor like NotePad++. According to this reference, it can handle files with 200Mb.

Use the regular expression [\h]+ to find for multiple spaces and then, replace with a delimiter (for example, "comma").

Just as an explanation:

  • "\h" will look for horizontal spaces without considering EOL (end of line),
  • "+" recognizes more than one whitespace in a sequence.

For more information see the tutorial: How to use regular expressions in Notepad++

I added a screenshot as a mini tutorial about how to proceed with this advice. Sorry for not being in English.

  1. Click the replace button
  2. Select option "Regular expression"
  3. Fill in Find = [\h]+ and Replace = , (comma)
  4. Click on "Replace all"

enter image description here

Save the new file an try to open it in Grass.

1

You can use "tr" as well:

cat yourfile | tr -s ' ' ' ' > newfile.csv

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.