3

I have a plugin, built with plugin builder 3.2.1. I'm running it in the current long-term release of QGIS, 3.16.13 using its built in version of Python 3.7. Within the run method, I call a second method that creates a set of point features:

def createPointLayer(self, crs, spacing)
    inset = spacing * 0.5
    vector_point_layer = QgsVectorLayer('Point', 'Name', 'memory', crs= crs)
    data_provider = vector_point_layer.dataProvider()

    x_min = self.dlg.extent.xMinimum() + inset #dlg is the dialog, extent is given by user through UI.
    x_max = self.dlg.extent.xMaximum()
    y_min = self.dlg.extent.yMinimum()
    y_max = self.dlg.extent.yMaximum() - inset

    points = []
    y = y_max
    while y  >= y_min:
        x = x_min
        while x <= x_max:
            geom = QgsGeometry.fromPointXY(QgsPointXY(x,y))
            feat = QgsFeature()
            feat.setGeometry(geom)
            points.append(feat)
            x += spacing
        y = y-spacing
        print('row is being created')
    data.provider.addFeatures(points)
    vector_points_layer.updateExtents()
    return vector_points_layer

This works fine, up until a certain number of points, when it will simply crash QGIS entirely without warning or feedback. QGIS will not shut down, it will completely stop responding with no way to stop it other than through the task manager (windows). It does not generate a crash report. It isn't a question of waiting, I have let it run for 10+ hours and it will not pick up again. I don't have a precise number of points for when on this happens, it happens around the same point each time if I keep the input the same, but if I change the input (extent, spacing) it will not crash at the same number of points (which I can see from the number of print statements I can see on screen once it crashes. Obviously I cannot scroll through any of the logs or print statements, since QGIS will have crashed, but I can see at which point it stopped printing). It does not freeze the rest of my computer, I can still do things in the background just fine. I have tried the following things:

  • Try, except (which does not catch it, it will still crash).
  • Manually delete the feature and geometry after they have been processed (see [https://gis.stackexchange.com/questions/82617/why-does-writing-large-shapefiles-crash-qgis])
  • Splitting up addFeatures() into adding every feature separately using addFeature() so that no list has to be used
  • Separating updateExtents() into multiple blocks, in case it just can't handle that number of updates.

My code is now the following with all methods above implemented:

def createPointLayer(self, crs, spacing)
    inset = spacing * 0.5
    vector_point_layer = QgsVectorLayer('Point', 'Name', 'memory', crs= crs)
    data_provider = vector_point_layer.dataProvider()

    x_min = self.dlg.extent.xMinimum() + inset #dlg is the dialog, extent is given by user.
    x_max = self.dlg.extent.xMaximum()
    y_min = self.dlg.extent.yMinimum()
    y_max = self.dlg.extent.yMaximum() - inset

    y = y_max
    while y  >= y_min:
        x = x_min
        while x<= x_max:
            try:
                geom = QgsGeometry.fromPointXY(QgsPointXY(x,y))
                feat = QgsFeature()
                feat.setGeometry(geom)
                del geom
                data_provider.addFeature(feat)
                del feat
                x += spacing
            except Exception as e:
                print(e, '\n', sys.exc_info()[0], ' occurred')
        print('row is being created')
        vector_layer_points.updateExtents()
        y = y-spacing
    return vector_points_layer

None of these methods work, it crashes at the same point. I've also tried various combinations of these methods, including placing the try, except differently within the method. It will still run fine for a low number of points, but crash QGIS for a high number. What is my issue?

UPDATE: The behaviour is now incredibly unstable. Clicking the QGIS window while the plugin is running appears to incite a crash, as does computer going into power saving mode. Most points I have managed to run before crashing is in the 80.000s, least in the 600s. This is with the same code as my second example, except I have added a counter that initiates before the first while loop and prints for every point.

9
  • Variables insert and inset are the same ?
    – xunilk
    Nov 15 '21 at 15:13
  • Yes, typo fixed. Thanks. Nov 15 '21 at 15:31
  • 1
    When QGIS stops printing stuff and "freezes" on heavy load, that is a normal behavior. Just wait until it finished without further interactions, the software is just busy. If you want to keep it responsive, take a look at background processing (tasks), but note that print() statements from a background process leads to an immediate crash.
    – MrXsquared
    Nov 15 '21 at 22:39
  • 3
    Now that I have read your question again in detail, I am pretty sure your whole issue is related to the use of print(). Remove it, reload and run your plugin, dont click anywhere in QGIS until its finshed. That should do it. Now that its most likely working, consider using background threads if you want to keep it responsive and make use of QgsMessageLog.logMessage("your print statement here","Name of the Message Panel",Qgis.Info) instead of print() (because print() will crash background threads as mentioned before). Maybe I can write a complete answer tomorrow...
    – MrXsquared
    Nov 15 '21 at 22:47
  • 1
    @MrXsquared This appears to have been the problem, my plugin finally ran until the end for 350.000 points with all print() statements replaced by QgsMessageLog.logmessage(). I have not applied background processes (yet). I will run an even bigger test over the weekend. If you find the time to turn your comment into an answer, I can award you the well deserved rep. Nov 19 '21 at 11:45
3

Your issue is the print() statement, which will lead to a freeze/crash. Remove it and you are fine. As an alternative you can use QgsMessageLog as for example QgsMessageLog.logMessage("My LogText", 'MyPlugin', level=Qgis.Info)

If you want to check yourself, you can run a basic code doing nothing but printing in a loop, like [print(i) for i in range(9999)] which will freeze QGIS until the process is done. If you do some heavy clicking on the GUI, chances are high that it will crash. But if you reduce the number to e.g. 99, its not heavy work and QGIS won't freeze.

From QGIS docs:

Use of the Python print statement is unsafe to do in any code which may be multithreaded and extremely slows down the algorithm. This includes expression functions, renderers, symbol layers and Processing algorithms (amongst others). In these cases you should always use the python logging module or thread safe classes (QgsLogger or QgsMessageLog) instead.

To keep your GUI responsive you need to use background threads. You can use QgsTask for this or also QThread or other methods to do this. Note that communicating from a background thread with the GUI directly will cause an immediate crash! So never ever use print() in a background thread!

The following quote from QGIS docs does not mention print(), but the same goes for it.

Any background task (regardless of how it is created) must NEVER use any QObject that lives on the main thread, such as accessing QgsVectorLayer, QgsProject or perform any GUI based operations like creating new widgets or interacting with existing widgets. Qt widgets must only be accessed or modified from the main thread. Data that is used in a task must be copied before the task is started. Attempting to use them from background threads will result in crashes.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.