I have a shapefile that consists of 375m x 375m cells from the State of California, consisting of a total of 3,022,037 cells. The raster file covers the same geographical area, but consists of smaller cells sized 50m x 50m. I am trying to run raster zonal statistics to infer aggregate features for each polygon in the shapefile.

  1. QGIS fails to complete this on a 2.5 GHz Intel Core i7 with 16 GB 1600 MHz DDR3 RAM. It freezes; the activity monitor shows that it goes to a non-responding state.

  2. A possible source for this problem is mentioned at Zonal Statistics Freezing QGIS 2.2. However, I ensured that the raster does not have any NA values, and there are no "complex" polygons in the shapefile.

  3. I also tried this with rasterstats, which abruptly ended after an hour. I am currently debugging to check what might have caused this.

  4. I also tried to explicitly code up the lookup in python myself, but naturally, this is significantly slower. It takes about 13-14 hours to work on smaller raster file.

My question seeks to know the standard procedure to deal with zonal statistics when the raster and vector files are large. This process needs to be repeated for 40-50 raster files, so this is a problem. A possible solution is mentioned in the paper "Distributed zonal statistics of big raster and vector data", but I am not sure if there is an open source code that can do this. Any guidance on this would be helpful.

  • 2
    One possibility to consider: convert the shapefile to a raster with the same alignment/resolution as the raster file. That would greatly simplify the zonal stats computations, but you'll lose some precision. – Jon Apr 28 '20 at 14:34
  • 1
    It may be better to do this in raster space rather than vector space. Even if you get around RAM limits the vector I/O will slow you down. I know of a way to do this but it involves software installation/learning curve so lets revisit if no better solution arises – Nathan Thomas Apr 28 '20 at 14:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.