QGIS crashes when creating many features through plugin

Question

I have a plugin, built with plugin builder 3.2.1. I'm running it in the current long-term release of QGIS, 3.16.13 using its built in version of Python 3.7. Within the run method, I call a second method that creates a set of point features:

def createPointLayer(self, crs, spacing)
    inset = spacing * 0.5
    vector_point_layer = QgsVectorLayer('Point', 'Name', 'memory', crs= crs)
    data_provider = vector_point_layer.dataProvider()

    x_min = self.dlg.extent.xMinimum() + inset #dlg is the dialog, extent is given by user through UI.
    x_max = self.dlg.extent.xMaximum()
    y_min = self.dlg.extent.yMinimum()
    y_max = self.dlg.extent.yMaximum() - inset

    points = []
    y = y_max
    while y  >= y_min:
        x = x_min
        while x <= x_max:
            geom = QgsGeometry.fromPointXY(QgsPointXY(x,y))
            feat = QgsFeature()
            feat.setGeometry(geom)
            points.append(feat)
            x += spacing
        y = y-spacing
        print('row is being created')
    data.provider.addFeatures(points)
    vector_points_layer.updateExtents()
    return vector_points_layer

This works fine, up until a certain number of points, when it will simply crash QGIS entirely without warning or feedback. QGIS will not shut down, it will completely stop responding with no way to stop it other than through the task manager (windows). It does not generate a crash report. It isn't a question of waiting, I have let it run for 10+ hours and it will not pick up again. I don't have a precise number of points for when on this happens, it happens around the same point each time if I keep the input the same, but if I change the input (extent, spacing) it will not crash at the same number of points (which I can see from the number of print statements I can see on screen once it crashes. Obviously I cannot scroll through any of the logs or print statements, since QGIS will have crashed, but I can see at which point it stopped printing). It does not freeze the rest of my computer, I can still do things in the background just fine. I have tried the following things:

• Try, except (which does not catch it, it will still crash).
• Manually delete the feature and geometry after they have been processed (see [https://gis.stackexchange.com/questions/82617/why-does-writing-large-shapefiles-crash-qgis])
• Splitting up addFeatures() into adding every feature separately using addFeature() so that no list has to be used
• Separating updateExtents() into multiple blocks, in case it just can't handle that number of updates.

My code is now the following with all methods above implemented:

def createPointLayer(self, crs, spacing)
    inset = spacing * 0.5
    vector_point_layer = QgsVectorLayer('Point', 'Name', 'memory', crs= crs)
    data_provider = vector_point_layer.dataProvider()

    x_min = self.dlg.extent.xMinimum() + inset #dlg is the dialog, extent is given by user.
    x_max = self.dlg.extent.xMaximum()
    y_min = self.dlg.extent.yMinimum()
    y_max = self.dlg.extent.yMaximum() - inset

    y = y_max
    while y  >= y_min:
        x = x_min
        while x<= x_max:
            try:
                geom = QgsGeometry.fromPointXY(QgsPointXY(x,y))
                feat = QgsFeature()
                feat.setGeometry(geom)
                del geom
                data_provider.addFeature(feat)
                del feat
                x += spacing
            except Exception as e:
                print(e, '\n', sys.exc_info()[0], ' occurred')
        print('row is being created')
        vector_layer_points.updateExtents()
        y = y-spacing
    return vector_points_layer

None of these methods work, it crashes at the same point. I've also tried various combinations of these methods, including placing the try, except differently within the method. It will still run fine for a low number of points, but crash QGIS for a high number. What is my issue?

UPDATE: The behaviour is now incredibly unstable. Clicking the QGIS window while the plugin is running appears to incite a crash, as does computer going into power saving mode. Most points I have managed to run before crashing is in the 80.000s, least in the 600s. This is with the same code as my second example, except I have added a counter that initiates before the first while loop and prints for every point.

Answer 1

Your issue is the print() statement, which will lead to a freeze/crash. Remove it and you are fine. As an alternative you can use QgsMessageLog as for example QgsMessageLog.logMessage("My LogText", 'MyPlugin', level=Qgis.Info)

If you want to check yourself, you can run a basic code doing nothing but printing in a loop, like [print(i) for i in range(9999)] which will freeze QGIS until the process is done. If you do some heavy clicking on the GUI, chances are high that it will crash. But if you reduce the number to e.g. 99, its not heavy work and QGIS won't freeze.

From QGIS docs:

Use of the Python print statement is unsafe to do in any code which may be multithreaded and extremely slows down the algorithm. This includes expression functions, renderers, symbol layers and Processing algorithms (amongst others). In these cases you should always use the python logging module or thread safe classes (QgsLogger or QgsMessageLog) instead.

To keep your GUI responsive you need to use background threads. You can use QgsTask for this or also QThread or other methods to do this. Note that communicating from a background thread with the GUI directly will cause an immediate crash! So never ever use print() in a background thread!

The following quote from QGIS docs does not mention print(), but the same goes for it.

Any background task (regardless of how it is created) must NEVER use any QObject that lives on the main thread, such as accessing QgsVectorLayer, QgsProject or perform any GUI based operations like creating new widgets or interacting with existing widgets. Qt widgets must only be accessed or modified from the main thread. Data that is used in a task must be copied before the task is started. Attempting to use them from background threads will result in crashes.