1

I'm using TINinterpolation in QGIS/pyQGIS to interpolate rasters from point-based model results.

Subsequent post-processing to produce a series of rasters containing index values at the same resolution and spatial extent in R produces result rasters that are much smaller (with individual files taking up 10-50% of the disk space). All of the rasters are stored on disk as GeoTIFF files.

Importing and exporting the input rasters in R using raster() then writeRaster() shrinks the filesize of the input rasters without noticeably changing the values. The re-processed rasters appear to be the same, and a couple of quick benchmark tests didn't show significant performance differences when reading them in to R using raster()

Are there settings within QGIS/pyQGIS to produce smaller rasters and avoid the extra step?

Is there a reason I should avoid re-processing and use the original larger files? I don't specify a compression, and do not think it defaults to a lossy compression.

The pyQGIS code I use to produce the rasters is:

layers = qgis.utils.iface.mapCanvas().layers()
feedback = QgsProcessingFeedback() 
for layer in layers:
    layerType = layer.type()
    ext = layer.extent()
    xMin = ext.xMinimum()
    xMax = ext.xMaximum()
    yMin = ext.yMinimum()
    yMax = ext.yMaximum()
    nCols = (xMax-xMin)/3.2808
    nRows = (yMax-yMin)/3.2808
    coords = "%f,%f,%f,%f [EPSG:2285]" %(xMin, xMax, yMin, yMax)
    if layerType == QgsMapLayer.VectorLayer:
        for i in range(0,len(use_cols)):
            col = use_cols[i]
            output_file_name = layer.name()+"_"+col_names[i]
            export_file_name = interim_path + output_file_name+".tif"
            vlayer = source_geopackage+"|layername="+layer.name()+"::~::0::~::"+str(col)+"::~::0"
            processing.run("qgis:tininterpolation", {
            'INTERPOLATION_DATA':vlayer,
            'METHOD':0,'COLUMNS':nCols,'ROWS':nRows,
            'EXTENT':coords,
            'OUTPUT': export_file_name},
            feedback = feedback)

Resulting rasters are 2.95 to 30.5 MB on disk

Using an example file that is 3.34MB on disk
Properties from QGIS:

Original bb60_2_291_Depth Name bb60_2_291_Depth Source D:/GIS/BarkleyBear/Model Results/bb60_2_20180806 - 1/bb60_2_rasters/bb60_2_291_Depth.tif Provider gdal CRS USER:100001 - * Generated CRS (+proj=lcc +lat_1=47.5 +lat_2=48.73333333333333 +lat_0=47 +lon_0=-120.8333333333333 +x_0=500000.0000000001 +y_0=0 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=us-ft +no_defs) - Projected Extent 1801231.2298138733021915,525557.9680938507663086 : 1805139.4590218793600798,534789.2082456786883995 Unit feet Width 1302 Height 3077 Data type Float64 - Sixty four bit floating point GDAL Driver Description GTiff GDAL Driver Metadata GeoTIFF Dataset Description D:/GIS/BarkleyBear/Model Results/bb60_2_20180806 - 1/bb60_2_rasters/bb60_2_291_Depth.tif Compression LZW Band 1 STATISTICS_MAXIMUM=15.99194996506 STATISTICS_MEAN=1.6378313050893 STATISTICS_MINIMUM=2.3352375055995e-006 STATISTICS_STDDEV=1.4764189154989 More information AREA_OR_POINT=Area Dimensions X: 1302 Y: 3077 Bands: 1 Origin 1.80123e+6,534789 Pixel Size 3.00171,-3.00008

In R reading this output raster in resaving it:

y <- raster(files_list[1])
writeRaster(y,paste0(output_folder,x)

and properties from QGIS for the resulting 1.63MB file:

Original bb60_2_291_Depth Name bb60_2_291_Depth Source D:/GIS/BarkleyBear/Model Results/bb60_2_20180806 - 1/bb60_2_rasters/reprocessed/bb60_2_291_Depth.tif Provider gdal CRS USER:100001 - * Generated CRS (+proj=lcc +lat_1=47.5 +lat_2=48.73333333333333 +lat_0=47 +lon_0=-120.8333333333333 +x_0=500000.0000000001 +y_0=0 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=us-ft +no_defs) - Projected Extent 1801231.2298138730693609,525557.9680938509991392 : 1805139.4590218788944185,534789.2082456790376455 Unit feet Width 1302 Height 3077 Data type Float32 - Thirty two bit floating point GDAL Driver Description GTiff GDAL Driver Metadata GeoTIFF Dataset Description D:/GIS/BarkleyBear/Model Results/bb60_2_20180806 - 1/bb60_2_rasters/reprocessed/bb60_2_291_Depth.tif Compression LZW Band 1 STATISTICS_MAXIMUM=15.99194996506 STATISTICS_MEAN=nan STATISTICS_MINIMUM=2.3352375055995e-006 STATISTICS_STDDEV=nan More information AREA_OR_POINT=Area Dimensions X: 1302 Y: 3077 Bands: 1 Origin 1.80123e+6,534789 Pixel Size 3.00171,-3.00008

The entire list, in this case ranges from 1.26 to 4.37 MB on disk, with a small list of rasters going from 93.8 MB to 23.4 MB.

  • Can you show the output of gdalinfo on large and small rasters? That will tell us if the resolution or bit depth has been changed and any compression algorithm used. gdalinfo is a command line utility but is also available in R from the gdalUtils package and is essentially what QGIS shows as the properties of a raster - you could cut and paste info from that instead. – Spacedman Apr 27 at 7:59
  • @Spacedman I updated with the properties from QGIS, and it does look like the bit depth changes from 64 bit floating point to 32 bit floating point. – Brian Fisher Apr 29 at 13:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.