I have a shapefile that consists of 375m x 375m cells from the State of California, consisting of a total of 3,022,037 cells. The raster file covers the same geographical area, but consists of smaller cells sized 50m x 50m. I am trying to run raster zonal statistics to infer aggregate features for each polygon in the shapefile.
QGIS fails to complete this on a 2.5 GHz Intel Core i7 with 16 GB 1600 MHz DDR3 RAM. It freezes; the activity monitor shows that it goes to a non-responding state.
A possible source for this problem is mentioned at Zonal Statistics Freezing QGIS 2.2. However, I ensured that the raster does not have any NA values, and there are no "complex" polygons in the shapefile.
I also tried this with rasterstats, which abruptly ended after an hour. I am currently debugging to check what might have caused this.
I also tried to explicitly code up the lookup in python myself, but naturally, this is significantly slower. It takes about 13-14 hours to work on smaller raster file.
My question seeks to know the standard procedure to deal with zonal statistics when the raster and vector files are large. This process needs to be repeated for 40-50 raster files, so this is a problem. A possible solution is mentioned in the paper "Distributed zonal statistics of big raster and vector data", but I am not sure if there is an open source code that can do this. Any guidance on this would be helpful.