14

I'm merging some shapefiles and I had some problems doing so inside QGIS, so i'm using ogr2ogr directly. I'm doing this (in a batch):

ogr2ogr -overwrite %destination% %n1%
ogr2ogr -update -append %destination% %n2% -nln all_new
ogr2ogr -update -append %destination% %n3% -nln all_new
ogr2ogr -update -append %destination% %n4% -nln all_new

It works fine, but now I need to have in the resulting shapefile, a field with the names of the original shapefiles I merged. Doesn't sound very difficult, but I'm not managing to do it.

Can anyone help? Thank you!

14

With small scripting it would be doable. With something like following you should be able to add column to a shapefile in all shapefiles in a folder, and merge them to merged.shp file

for %f in (*.shp) do (
  ogrinfo %f -sql "ALTER TABLE %f ADD COLUMN filename character(15)"
  ogrinfo %f -sql "UPDATE TABLE %f filename = '%f'"
  ogr2ogr -update -append merged.shp %f -f “esri shapefile” -nln merge 
)

edit: The same as a Bash script, with some changes to make it working:

for f in *.shp
do 
  base=${f%.shp}
  ogrinfo $f -sql "ALTER TABLE $base ADD COLUMN filename character(15)"
  ogrinfo $f -dialect SQLite -sql "UPDATE $base SET filename = '$base'"
  ogr2ogr -update -append merged.shp $f
done
  • This solution would only work if all of your layers/shapefiles had the same name e.g. "CHEMIN" correct? I'm looking for a solution for scripting files with different layer names. – oeon Nov 29 '12 at 15:13
  • Correct, if you replace CHEMIN with %f then it should work with any name, as in shapefile table name is layer name. I edited answer. I do not have windows to test it really – JaakL Dec 1 '12 at 21:02
9

I would use the -sql option, and import the shapefile in the following way:

ogr2ogr -update -append %destination% %n2% -sql 'SELECT "%n2%" as SHAPE_ORIG, field1, field2, ... FROM %n2%'
  • Paolo - I'm struggling to get this to work. Can it only be done with ogr2ogr and not ogrinfo? I've posted to gdal-dev too, with my examples lists.osgeo.org/pipermail/gdal-dev/2012-November/034849.html Can't do it in Windows or Bash.. – oeon Nov 29 '12 at 16:01
  • Joe, does the single ogrinfo line fails, or it is failing only in the context of the script (inside the loop)? – capooti Nov 29 '12 at 20:08
  • Paolo - I am going to add the answer that worked for me, below. Thank you for following up with me, kind Sir! :) – oeon Dec 1 '12 at 2:24
7

there are some ways for merging shapefiles.

  • if you want to merge layers as a one layer, you can use MMqgis tools for merging...

mmqgis

  • if you want to merge all shapefiles under a folder, you can use DARREN COPE simple code here.

mkdir merged
for %f in (*.shp) do (
if not exist merged\merged.shp (
ogr2ogr -f “esri shapefile” merged\merged.shp %f) else (
ogr2ogr -f “esri shapefile” -update -append merged\merged.shp %f -nln Merged )
)
  • beside this can use GeoMerge free tool for merging lots of file but dont forget to consider your file size for workin with it.

and adding attribute to shapefile @dango directon is good. you can use layer.CreateField(field_name) for creating a new column which is populated from

import os
shapeFileName = os.path.splitext("your_shape_file_path")[0]

i hope it helps you...

5

vascobnunes, here's how I accomplished this problem by using a Python script to daisy-chain several ogr2ogr instructions together. You could easily convert it to a batch script, basically I just concatenate together ogr2ogr instructions (cmd), then execute them by calling os.system(cmd), passing-in the ogr2ogr command I concatenated together.

The secret weapon is (as capooti demonstrated) applying OGR_SQL to impose the filename as a constant value of the source dataset you are appending into your merge result.

In my example, the -sql flag handles this, in the code it's like this:

-sql "SELECT \'' + filename + '\' AS filename, * FROM ' + filenameNoExt + '"'

But that's confusing to read because I need to apply single quotes and double quotes in the resulting concatenation. To do that I have to escape the single quotes (i.e. \') to use them "for real". So for readability, it helps to see it without variables and escape sequences. If you pretend the filename was "roads1" for a particular iteration, the resulting concatenation would look like this in the ogr2ogr sentence:

-sql "SELECT 'roads1.shp' AS filename, * FROM roads1"

This .py script is an amalgamation of three tricks I stole from matt wilkie (an empty, clone of a shapefile), j03lar50n (adding a column to a shapefile using ogrinfo and ogr_sql), and capooti (using ogr_sql to impose a fixed column value on all records in a shapefile). So here's the full script:


# merge_shps.py
import os    

path = "D:/GIS/01_tutorials/ND_Roads/extracted"  # path to your folder of .shp files
merge = "merge_filename"                         # this will be the name of your merged result

directory = os.listdir(path)

count = 0
for filename in directory:
    if ".SHP" in filename.upper() and not ".XML" in filename.upper():

        # On the first pass, create a clone and add the filename column.
        if count == 0:
            # Make a clone (matt wilkie)..
            cmd = 'ogr2ogr ' + path + '/' + merge + '.shp ' + path + '/' + filename + ' -where "FID < 0"'
            os.system(cmd)

            # Add the field (j03lar50n)..
            cmd = 'ogrinfo ' + path + '/' + merge + '.shp -sql "ALTER TABLE ' + merge + ' ADD COLUMN filename character(50)"'
            os.system(cmd)

        # Now populate the data (capooti)..
        print "Merging: " + str(filename)

        # You'll need the filename without the .shp extension for the OGR_SQL..
        filenameNoExt = filename.replace(".shp","")

        cmd = 'ogr2ogr -f "esri shapefile" -update -append ' + \
                path + '/' + merge + '.shp ' + \
                path + '/' + filename + \
                ' -sql "SELECT \'' + filename + '\' AS filename, * FROM ' + filenameNoExt + '"'

        # Uncomment this line to spit the ogr2ogr sentence to the terminal..
        #print "\n" + cmd + "\n"

        os.system(cmd)

        count += 1
4

Add column with source filename from folder of shapefiles. Requires GDAL 1.10dev, my attempt to drop .shp extension isn't working - but overall, works. - I imagine it could be added to the lines that do merging with OGR.

for f in *.shp;

do

name=${f%.shp}

/Users/you/gdal_src/bin/ogrinfo $f -sql "ALTER TABLE $name ADD COLUMN filename character(21)"
/Users/you/gdal_src/bin/ogrinfo $f -dialect SQLite -sql "UPDATE $name SET filename = '$f'"
done;
  • +1 for the dialect specification. I was getting errors on the top answer due to OGR SQL not doing updates. – user15741 Apr 9 at 15:41
3

Hi maybe this link will help. It shows how to add a feild to a shapefile using the python gdal bindings.

We're looking for long answers that provide some explanation and context. Don't just give a one-line answer; explain why your answer is right, ideally with citations. Answers that don't include explanations may be removed.

2

Inside QGIS you can add the Merge Shapefile plugin. There is an option to "Add column with file name"enter image description here

  • I get a TypeError: object of type 'NoneType' has no len() – Hannes Ledegen Jan 3 '17 at 9:24
0

A slightly modified version of JaaKL's answer. Note that the -append foo.shp and -nln foo need to match. Also, note the use of the SQLite dialect (GDAL apparently doesn't accept the keyword 'Update', so the SQLite dialect must be used instaed), and the absence of the keyword 'TABLE' after the word 'UPDATE' (not needed or accepted by SQLite).

for %%f in (*.shp) do (
  if not "%%f" == "merge.shp" (
    ogrinfo %%f -sql "ALTER TABLE %%~nf ADD COLUMN fname character(15)"
    ogrinfo %%f -dialect SQLite -sql "UPDATE %%~nf SET fname = '%%~nf'"
    ogr2ogr -update -append merge.shp %%f -f "ESRI SHAPEFILE" -nln merge 
  )
)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.