Take the 2-minute tour ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.

I am working on scripting some data processing using Qgis and Python, but I'm running across issues that I need to debug. Since my data is really large, and run-times can be lengthy, I would like to be able to write out results periodically and then append to those results so I don't have to re-run my scripts.

I'm outputting a polygon shape file, and I am using the following python code currently to work with the shp file:

poly_provider = poly_layer.dataProvider()

fields = { 0 : QgsField("STAT_NO", QVariant.String) }

writer = QgsVectorFileWriter("/localstore/data/shps/contrib_area.shp", "CP1250", fields, poly_provider.geometryType(), poly_layer.crs(), "ESRI Shapefile")

if writer.hasError() != QgsVectorFileWriter.NoError:
  print "Error when creating shapefile: ", writer.hasError()

#processing...

def collect_geoms(geometries):
  if len(geometries) > 0:
    union_geoms = geometries[0]
    for geometry in geometries:
      union_geoms = union_geoms.combine(geometry)

    fet = QgsFeature()
    fet.setGeometry(union_geoms)
    fet.addAttribute(0, QVariant( point_feat_dict[point] ))
    writer.addFeature(fet)

The collect_geoms() function is then called in a loop, and when I'm done, I use del writer and I get the shape file written to the disk.

Is there a way that I can write the shape file out incrementally and append new polygons to it? So for instance I can run 10 iterations of the loop, write the shape file, and then run the next ten without overwriting the first set?

share|improve this question
add comment

1 Answer

up vote 1 down vote accepted

This is how you could do it:

shp = "/localstore/data/shps/contrib_area.shp"
writer = QgsVectorFileWriter(shp, "CP1250", fields, poly_provider.geometryType(), poly_layer.crs(), "ESRI Shapefile")
# Just nuke the writer because we have already made our empty shapefile on disk.
del writer

# Open a reference to the new layer
layer = QgsVectorLayer(shp, "contrib_area","ogr")
dp = layer.dataProvider()

# in your process block
features = get10Features()
dp.addFeatures(features)

QgsVectorDataProvider::addFeatures() will write directly to the disk/datasource so that is the only method you should need to call.

share|improve this answer
    
I hadn't thought about that. I spent a lot of time digging through the writer documentation trying to find a good solution. I'll try this out and see if it works, might even reduce my memory footprint if addFeatures() writes to disk. –  Nelz11 Jun 18 '13 at 3:29
    
QgsVectorFileWriter is a bit ugly IMO. There really should just be a save() method on the QgsVectorLayer class. –  Nathan W Jun 18 '13 at 5:24
    
I agree. On a similar note, @Nathan, do you know how efficient combining geometries is with Qgis? Seems like when I get a large number of geometries it tries to hang (or it just takes a long time--several days!) This is what is leading me to try to only do parts of my loop at a time to figure out where and why it is taking so long... –  Nelz11 Jun 18 '13 at 15:35
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.