I'm very new to QGIS, PyQt and even Python so I'm sorry if there is a very obvious mistake I am making, but I can seem to get one of my functions to run from within a plugin I'm writing for QGIS (3.16). When I run it directly from a test main as a script it functions perfectly fine. Ultimately, I would like to run it on a separate thread but for now, it even crashes QGIS when run on the main thread. (it also crashes QGIS when ran from a separate thread, I have tried it with QgsTask.fromFunction and TaskManager as well as PyQt5's ThreadPool
and QRunnable
subclass).
What I'm wondering about is the following: when I call the exact same function from a test main that I made, from within a virtual environment that is exactly the same as the QGIS Python setup, it runs just fine. I can obviously share some of my code, but I was wondering whether that fact alone pointed to something I'm missing. The line of code that seems to be causing the crash calls geopandas sjoin
.
Further relevant information:
Both dataframes involved in the sjoin
are created within the function that I want to run, so I doubt it's a matter of scope?
I have a general try/except clause set up that should catch (and currently print and ignore) all exceptions. However, that doesn't seem to prevent the crash.
The structure of the code:
pluginclass.run()
button.clicked.connect(parent_of_crasher)
parent_of_crasher calls crasher
crasher calls subfunction_of_crasher
In my test main, I just call the function directly from main.
One argument is being passed to the crashing function (a python class object), but it's passed to every function I call from the parent and hasn't created any issues anywhere else, so I'm pretty sure that isn't the problem.
Where it crashes:
#(fixeddrainage and conn_nodes_geo are both geodataframes passed into the subfunction_of_crasher by the 'crasher')
try:
joined = sjoin(
fixeddrainage, conn_nodes_geo, how='left', op='intersects', lsuffix='fd', rsuffix='conn')
except Exception as e:
QgsMessageLog(e)
raise e from None
#After testing it on a hunch: it also crashes on
overl = overlay(fixeddrainage, conn_nodes_geo, how="intersection")
I can print and modify both fixeddrainage
and conn_nodes_geo
without crashing QGIS, which leads me to believe it's not a memory/scope issue (I tried it by renaming columns in place, and doing a groupby). Furthermore, it never prints an exception to the message log, even when I pass instead of raising the exception.
It does seem to only happen with geopandas functions.
I am really at a loss here, so any insight is very welcome! I can provide more detailed information if that would help, but I figured I would describe the outline in case it points to a common mistake!
Edit: geopandas version is 0.8.1 in case that is relevant.
Edit: I have been experimenting further and I have found the following does work:
for index, row in fixeddrainage.iterrows():
nodes_in_area = conn_nodes_geo.intersection(row.geometry)
nodes_in_area_index = nodes_in_area[~nodes_in_area.is_empty].index
fixeddrainage.loc[index, 'area_nodes_m2'] = round(
conn_nodes_geo.loc[nodes_in_area_index, 'storage_area'].sum(), 0)
While this:
joined = sjoin(
fixeddrainage, conn_nodes_geo, how='left', op='intersects', lsuffix='fd', rsuffix='conn')
group = joined.groupby([peil_id, 'multipolygon_level'])[storage_area].sum()
fixeddrainage = fixeddrainage.merge(group, how='left', on=[peil_id, 'multipolygon_level'])
However, I would rather not use the iterrows method. Can anyone tell me what the difference is (keeping in mind that the function executes fine outside of qgis) between the two methods?