0

This is the scenario: I have thousands of shapefiles organized in different folders. My purpose is to merge all the shapefiles with the same name into a new single shapefile. I know I can import all of them into QGIS and merge them manually with MMQGIS plugin, but I was thinking if it would be possible to automate the operation with a script, as doing it manually would be extremely time consuming...

  • does it have to be inside QGIS? otherwise you could use ogr2ogr and a script – Ian Turton Mar 3 '16 at 21:14
  • @iant I never used ogr2ogr so if it's possible to do the same thing with a script in QGIS I would prefer... – Bryan Ciceri Mar 3 '16 at 21:39
  • Personally I would just search with your file browser and copy the found files into a new folder and merge them. You could also use a union VRT to merge them which works quite nicely even on vectors. – Sethinacan Mar 4 '16 at 11:17
  • @Sethinacan I'm looking for a faster way to do this operation cause I have to do this with thousands files... – Bryan Ciceri Mar 4 '16 at 13:05
  • Not just thousands of files but lots of sets with the same name? then a recursive script is definitely a better bet. – Sethinacan Mar 4 '16 at 14:30
1

I guess it's totally possible using python scripting in QGIS but it may not be the fastest solution (or at least my answer, involving some nested loop isn't probably the more efficient answer, as there is probably a way to merge directly two (or more) layers in QGIS).

Anyway, if it can helps :
- in a first time it walks in your folders, fetching the path of each file with the same name.
- in a second time its create a new shapefile, regrouping the features of each shp with the same name.

import os

seen_name = {}  # Instanciate a dict, the key will be the name and the value each real path
for dirpath, dirnames, filenames in os.walk('/tmp/geo'):
    for filename in filenames:
        if '.shp' in filename:
            if filename in seen_name:
                seen_name[filename].append(os.path.join(dirpath, filename))
            else:
                seen_name[filename] = [os.path.join(dirpath, filename)]


os.chdir(Your_new_clean_folder)

for unique_name, list_filepath in seen_name.items():
    ref_lyr = QgsVectorLayer(list_filepath[0], unique_name, 'ogr')
    features = []

    for nb, unique_file in enumerate(list_filepath[1:]):
        lyr = QgsVectorLayer(unique_file, unique_name+str(nb), 'ogr')
        features.extend([ft for ft in lyr.getFeatures()])

    ref_lyr.dataProvider().addFeatures(features)
    QgsVectorFileWriter.writeAsVectorFormat(
        ref_lyr, unique_name, "UTF8", ref_lyr.crs(), "ESRI Shapefile")

I tested on a folder containing some nested folders (at various levels) randomly containing a bunch of shapefiles with the same name dispatched in these folders, its seems to work fine (I edited for an answer taking slightly less time, but if you have many layers, with many features it will take long as it loops on each features..).

Otherwise you might have a look to the processing module and the 'qgis:mergevectorlayers' alg. it provides (but it seems to only take two layers).

  • what folders does the script walk through? Every folder in my drive? Furthermore I was thinking if it could be a better approach to import all the layers I would like to merge into QGIS and then launch a script that scans the imported layers list to create an array of unique names, and then invokes the MMQGIS merge layers plugin for any index of the array, selecting the layers with the same name of the index value from the imported layers – Bryan Ciceri Mar 4 '16 at 10:11
  • The os.walk function is aimed to traverse the whole directory tree starting from the path you give to it (ie. here it only walks in /tmp and its subfolders). However, the steps you are mentioning are basically what this script is doing (without MMQGIS though). – mgc Mar 4 '16 at 17:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.