2

I'm trying to write large shapefiles with QGIS 2.0 API, using Python, which crashes QGIS. The data is composed of simple two point lines and a data table of five columns. A thousand features or so works fine, but at 10k the program crashes for sure. Perhaps some sort of writing in chunks is a solution, or the problem lies in my code :


def write_lines (file_name, data_list, coordinate_ref_system):

    fields = QgsFields() 
    fields.append(QgsField("Data_1", QVariant.String ))
    fields.append(QgsField("Data_2", QVariant.String))

    writer = QgsVectorFileWriter( file_name + ".shp", "CP1250", fields,
                          QGis.WKBLineString, coordinate_ref_system)
    if writer.hasError() != QgsVectorFileWriter.NoError:
        QMessageBox.information(None, "ERROR!", "Cannot write file (?)")
        return 0

    for r in data_list:
        # create a new feature
        feat = QgsFeature()

        l_start=QgsPoint(r[0],r[1])
        l_end =QgsPoint(r[2],r[3])
        feat.setGeometry(QgsGeometry.fromPolyline([l_start, l_end]))

        feat.setFields(fields)
        #feat.setAttributes([ str(r[4]), str(r[5]) ]) DOESN'T WORK?

        feat['Data_1'] = r[4]
        feat['Data_2'] = r[5]

        writer.addFeature(feat)
    del writer

Thanks!

  • Do you have details of the crash? Also, can you try to fix the formatting on your code? – BradHards Jan 11 '14 at 10:38
  • QGIS freezes et the moment of writing to disk, and just shuts down after a while: no exit message... Sorry for the code, I'm trying to fix it (a newbie...) – Zoran Jan 11 '14 at 13:03
  • 2
    Can you try to run QGIS under a debugger or valgrind? Also, maybe try del feat within the for r in data_list loop – BradHards Jan 11 '14 at 22:02
3

Adding content of comment against question as an answer to allow this to be closed

In the loop, the construction of the feature

    for r in data_list:
        # create a new feature
        feat = QgsFeature()
        # do something with the new feature
        ....

Is allocating memory for the feature, but it isn't being released anywhere. Conceptually the python memory manager should be able to tell that the feature could be garbage collected, but that is often difficult in applications that wrap C/C++ calls.

The solution to running out of memory in this case is to provide an explicit "I've finished with that feature, thanks" indication, using del

So the code structure will look like:

    for r in data_list:
        # create a new feature
        feat = QgsFeature()
        # do something with the new feature
        ....
        del feat

Note: This technique is generally applicable (so it might be regarded as off-topic for GIS.SE), but since you wouldn't need to do this in most python programming, I think its "close enough"

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.