Abrupt error when trying to run model with viewshed as batch process

Question

I am trying to run the following model as a batch process. In summary, I take a DEM raster layer and a number of individual points as separate .gpkg files in a directory (these points have been created using the 'Create viewpoints' function in the Visibility Analysis plugin. I then generate the binary viewshed of each point, vectorize it into a polygon, dissolve it based on the visibility parameter (1 or 0) and extract only the visible (1) areas and save the resulting polygon in another directory.

When trying to run the model as a batch process on both QGIS 3.32.0 RC and QGIS 3.28.8 LTR, the first few files run successfully and generate the correct result. For example,

Processing algorithm 1/865…
Algorithm site_viewshed_vector_generation starting…
Input parameters:
{"dem":"C:/Users/Work/Projects/Building_Cover_Affected_Sites/data/processed/Building_Elevation_Combined_DEM.tif","site_viewpoints":"C:/Users/Work/Projects/Building_Cover_Affected_Sites/data/processed/Site_Viewpoints/CM0001.gpkg","visible_viewshed":"C:/Users/Work/Projects/Building_Cover_Affected_Sites/data/processed/Site_Viewsheds/view_CM0001.gpkg"}

Model processed OK. Executed 4 algorithm(s) total in 97.32 s.
Algorithm site_viewshed_vector_generation correctly executed…
Execution completed in 97.329 seconds
Results:
{"CHILD_INPUTS":{"native:dissolve_1":{"FIELD":["VIEW"],"INPUT":"Vector_polygons_1a077f13_d80b_4dd9_b32c_d52219f44cf6","OUTPUT":"memory:Dissolved","SEPARATE_DISJOINT":false},"native:extractbyexpression_1":{"EXPRESSION":"\"VIEW\" = 1","INPUT":"Dissolved_0b4b1672_183f_4328_b1bd_ba125a739201","OUTPUT":""},"native:pixelstopolygons_1":{"FIELD_NAME":"VIEW","INPUT_RASTER":"C:/Users/Work/AppData/Local/Temp/processing_ukeBuJ/9a76815a7ac84c2e8e472e8aa9f087f4/OUTPUT.tif","OUTPUT":"memory:Vector polygons","RASTER_BAND":"1"},"visibility:viewshed_1":{"ANALYSIS_TYPE":0,"DEM":"Building_Elevation_Combined_DEM_9e501972_ac6d_4f8a_90bd_30cfdd5fc8c0","OBSERVER_POINTS":"C:/Users/Work/Projects/Building_Cover_Affected_Sites/data/processed/Site_Viewpoints/CM0001.gpkg","OPERATOR":0,"OUTPUT":"C:/Users/Work/AppData/Local/Temp/processing_ukeBuJ/9a76815a7ac84c2e8e472e8aa9f087f4/OUTPUT.tif","REFRACTION":0.13,"USE_CURVATURE":true}},"CHILD_RESULTS":{"native:dissolve_1":{"OUTPUT":"Dissolved_0b4b1672_183f_4328_b1bd_ba125a739201"},"native:extractbyexpression_1":{"OUTPUT":"C:/Users/Work/Projects/Building_Cover_Affected_Sites/data/processed/Site_Viewsheds/view_CM0001.gpkg"},"native:pixelstopolygons_1":{"OUTPUT":"Vector_polygons_1a077f13_d80b_4dd9_b32c_d52219f44cf6"},"visibility:viewshed_1":{"OUTPUT":"C:/Users/Work/AppData/Local/Temp/processing_ukeBuJ/9a76815a7ac84c2e8e472e8aa9f087f4/OUTPUT.tif"}},"visible_viewshed":"C:/Users/Work/Projects/Building_Cover_Affected_Sites/data/processed/Site_Viewsheds/view_CM0001.gpkg"}

However, after a certain point it starts to fail with the following message:

Processing algorithm 12/865…
Algorithm site_viewshed_vector_generation starting…
Input parameters:
{"dem":"C:/Users/Work/Projects/Building_Cover_Affected_Sites/data/processed/Building_Elevation_Combined_DEM.tif","site_viewpoints":"C:/Users/Work/Projects/Building_Cover_Affected_Sites/data/processed/Site_Viewpoints/CM0024.gpkg","visible_viewshed":"C:/Users/Work/Projects/Building_Cover_Affected_Sites/data/processed/Site_Viewsheds/view_CM0024.gpkg"}

Error encountered while running Viewshed
Algorithm site_viewshed_vector_generation failed…
Execution failed after 0.042 seconds

The faliure cascades from the point it first fails onwards. Not sure why this happens.

Answer 1

I realized the issue. Turns out, during the Viewshed step of the above model the algorithm generates a temp raster file of approx 8 GB per point that I run it for. This temp file is not deleted until QGIS is closed. Hence, the process fails once there is insufficient storage.

Resolved this by moving the task to a server with sufficient storage capacity and by breaking points into multiple batches.