QGIS crashes with "bad_alloc" when processing NetCDF file using xarray

Question

I need to merge multiple NetCDF files in a QGIS plugin I am developing. I'm using xarray to do so, however, QGIS crashes with the error "Critical: Caught unhandled std::exception: std::bad_alloc".

If I use the same script, but I launch it using Visual Studio Code, it works fine.

I have checked the RAM usage of my computer when the script is run from both QGIS and vscode and it barely goes up with still at least 20GB of available RAM (for computer 1 below).

The script looks like this:

import xarray
ds = xarray.open_mfdataset('/home/francis/Documents/ncfiles/*tmin.nc', parallel=False)

xarray version is 2023.10.1, but I also tried with version 2022.11.0.

I tried the following computers:

1. QGIS 3.28 and 3.33.0-Master, Fedora Linux 38, 32GB RAM
2. QGIS 3.28, Fedora Linux 38, 8GB RAM
3. QGIS 3.32.3-Lima, MacOS, 64 GB RAM

Computer 1 always crashed, computer 2 was successful for a while, then it started to crash too. Computer 3 works fine.

Is there something else I could try to merge multiple NetCDF files without QGIS crashing? I'd like to stick with xarray if possible.

Edit: I just tried with open_dataset() using one netCDF file of 1MB and it crashes QGIS with the same error message.

Answer 1

Apparently it was a problem with the installation of xarray. I had to use Fedora's package manager instead of using pip.