5

I have a few hundred txt files (each representing a line of a river bed profile) that I need to load as (point-) layers in QGIS. Partial example:

4504764.8331;5374251.3024;397.8871;2
4504764.7761;5374250.8056;397.8555;2
4504764.7191;5374250.3089;397.8295;2
4504764.6621;5374249.8121;397.8149;2
4504764.6051;5374249.3154;397.7977;2
4504764.5481;5374248.8186;397.7725;2

It works manually (layer -> add delimited text layer, then renaming the columns after import) but given the volume I can't do it all by hand.

I found a similar two year old question here: How to import multiple textfiles in one step , but "create a plain text file called .vrt file for each of csv files" doesn't sound so peachy. The files are also missing the header line, but I know the format: "X; Y; Z; classification".

Is there any way to do this without having to code up a text generator for those vrt files and inserting header lines into all csvs?

  • 1
    Don't have time to flesh out a full answer, but I have a blog post which goes through automatically adding a header to the start of each file, and automatically creating the .vrt for each file at (this is in Linux though, so will require work for other OS's): gisforthought.com/… – HeikkiVesanto May 8 '15 at 14:19
  • 2
    Instead of a separate .vrt file for each .csv it should be possible since GDAL 1.10.0 to have only one .vrt with an union layer gdal.org/drv_vrt.html. I am not sure if that is easier to create or not - with .csv it would mean an union layer made of a bunch of other vrt files. – user30184 May 8 '15 at 14:25
8

Assuming all your text files are into the same directory, you can run this code snippet in the QGIS Python console to get your files loaded as individual layers in QGIS:

import os.path, glob
layers=[]
for file in glob.glob('/tmp/xy/*.txt'): # Change this base path
  uri = "file:///" + file + "?delimiter=%s&xField=%s&yField=%s&useHeader=no&crs=epsg:3116" % (";", "field_1","field_2")
  vlayer = QgsVectorLayer(uri, os.path.basename(file), "delimitedtext")
  vlayer.addAttributeAlias(0,'X')
  vlayer.addAttributeAlias(1,'Y')
  vlayer.addAttributeAlias(2,'Z')
  vlayer.addAttributeAlias(3,'classification')
  layers.append(vlayer)

QgsMapLayerRegistry.instance().addMapLayers(layers)

As you see, it works for GNU/Linux paths, I don't know how the uri (specifically the file path) should look like for Windows. Also, adjust the crs for your own case.

I also wanted to tell you how to change field names automatically, but it seems that delimited-text based layers cannot be edited. I guess the most you can do is to assign aliases for your columns, as the code does.

  • I solved my initial problem with some extra data I managed to get. But your solution looks like it would be the better way. However, I couldn't get it to work ("Errors in file / File cannot be opened or delimiter parameters are not valid" for every file). It's probably due to the path? I changed the path stuff but with little success: for file in glob.glob('C:\\temp\\linetest\\*.txt'): # Change this base path uri = file + "?delimiter=%s&xField=%s&yField=%s&useHeader=no&crs=epsg:31468" % (";", "field_1","field_2") print(uri). – MapEngine May 28 '15 at 14:52
  • (comment too long) This is the print result: C:\temp\linetest\test_line_5.txt?delimiter=;&xField=field_1&yField=field_2&useHeader=no&crs=epsg:31468. Pasting "C:\temp\linetest\test_line_5.txt" into the explorer bar opens the file however. – MapEngine May 28 '15 at 14:54
  • Yes, I gues the problem is related to Windows paths and the "file:///" prefix QGIS expects. In a couple of days I'll test paths on Windows and let you know the proper way of doing it. – Germán Carrillo May 28 '15 at 15:47
  • 1
    That's it - we just need to specify "file:///" with all three slashes. The rest of the script works like it is in Windows, no need for the escaped backslashes in the base path either. Thanks! – MapEngine May 29 '15 at 8:10
  • That's great! I've edited the post to add the third slash in the script. – Germán Carrillo May 29 '15 at 13:34
3

Combine all text files in a windows directory with the copy command:

copy *.txt river.txt

On Linux use the cat (concatenate) command:

cat * > river.txt

Then in QGIS for the merged text file use 'Add Delimited Text Layer' button.

  • 1
    This answer is correct, if you want a combined result. However my question was edited to ask for a single layer from multiple text files, which wasn't what I asked about initially. – MapEngine May 11 '15 at 0:08
3

The answer from @GermanCarrillo worked for me and saved much time.

With a text file structured like so: Time[s],Easting[m],Northing[m],Height[m],Roll[deg],Pitch[deg],Yaw[deg] 170258.002391,332731.5085,5794908.7794,46.1642,0.409501631306,-5.248541020385,0.684043081908

The code that worked for me in windows is:

import os.path, glob
layers=[]
for file in glob.glob('C:/Projects/2829/locations/20160524/*.txt'): # Change this base path
  uri = "file:///" + file + "?delimiter=%s&xField=%s&yField=%s&useHeader=no&crs=epsg:28355" % (",", "field_2","field_3")
  vlayer = QgsVectorLayer(uri, os.path.basename(file), "delimitedtext")
  vlayer.addAttributeAlias(1,'X')
  vlayer.addAttributeAlias(2,'Y')
  layers.append(vlayer)

QgsMapLayerRegistry.instance().addMapLayers(layers)
1

I would like to add something to Germán Carrillo answer. For windows and QGIS 2.18.9 his code works perfectly for me. I just added a line of code to identify the path on windows:

pathName = r'put/your/path/here/spearated/by/slashes/'

you can copy+paste your path from the windows folder and change backslashes \ to slashes / before copying it to pathName.

Then, you just change the for line like follows

for file in glob.glob(pathName+'*.txt'): 

Hope this helps

1

from @nitrofruit @GermánCarrillo

Modified for QGIS 3:

import os.path, glob
from qgis.core import QgsProject, QgsVectorLayer
layers=[]
for file in glob.glob('C:/filepath/*.txt'): # Change this base path and change backslashes to forward slashes
  uri = "file:///" + file + "?delimiter=%s&xField=%s&yField=%s&useHeader=no&crs=epsg:28355" % (",","field_2","field_3") #(Delimeter, XField, YField) #1st column in file is column 1
  vlayer = QgsVectorLayer(uri, os.path.basename(file), "delimitedtext")
  vlayer.setFieldAlias(1,'X') #these order of columns go: 0, 1, 2 etc
  vlayer.setFieldAlias(2,'Y')
  layers.append(vlayer)

QgsProject.instance().addMapLayers(layers)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.