I am working on scripting some data processing using Qgis and Python, but I'm running across issues that I need to debug. Since my data is really large, and run-times can be lengthy, I would like to be able to write out results periodically and then append to those results so I don't have to re-run my scripts.

I'm outputting a polygon shape file, and I am using the following python code currently to work with the shp file:

poly_provider = poly_layer.dataProvider()
 
fields = { 0 : QgsField("STAT_NO", QVariant.String) }

writer = QgsVectorFileWriter("/localstore/data/shps/contrib_area.shp", "CP1250", fields, poly_provider.geometryType(), poly_layer.crs(), "ESRI Shapefile")

if writer.hasError() != QgsVectorFileWriter.NoError:
  print "Error when creating shapefile: ", writer.hasError()

#processing...

def collect_geoms(geometries):
  if len(geometries) > 0:
    union_geoms = geometries[0]
    for geometry in geometries:
      union_geoms = union_geoms.combine(geometry)

    fet = QgsFeature()
    fet.setGeometry(union_geoms)
    fet.addAttribute(0, QVariant( point_feat_dict[point] ))
    writer.addFeature(fet)

The collect_geoms() function is then called in a loop, and when I'm done, I use del writer and I get the shape file written to the disk.

Is there a way that I can write the shape file out incrementally and append new polygons to it? So for instance I can run 10 iterations of the loop, write the shape file, and then run the next ten without overwriting the first set?

I am working on scripting some data processing using QGIS and Python, but I'm running across issues that I need to debug. Since my data is really large, and run-times can be lengthy, I would like to be able to write out results periodically and then append to those results so I don't have to re-run my scripts.

I'm outputting a polygon shapefile, and I am using the following Python code currently to work with the shp file:

poly_provider = poly_layer.dataProvider()    
fields = { 0 : QgsField("STAT_NO", QVariant.String) }

writer = QgsVectorFileWriter("/localstore/data/shps/contrib_area.shp", "CP1250", fields, poly_provider.geometryType(), poly_layer.crs(), "ESRI Shapefile")

if writer.hasError() != QgsVectorFileWriter.NoError:
  print "Error when creating shapefile: ", writer.hasError()

#processing...

def collect_geoms(geometries):
  if len(geometries) > 0:
    union_geoms = geometries[0]
    for geometry in geometries:
      union_geoms = union_geoms.combine(geometry)

    fet = QgsFeature()
    fet.setGeometry(union_geoms)
    fet.addAttribute(0, QVariant( point_feat_dict[point] ))
    writer.addFeature(fet)

The collect_geoms() function is then called in a loop, and when I'm done, I use del writer and I get the shapefile written to the disk.

Is there a way that I can write the shapefile out incrementally and append new polygons to it? So for instance I can run 10 iterations of the loop, write the shapefile, and then run the next ten without overwriting the first set?