Insufficient memory available for raster calculator on DEM with float 32 pixel type

Question

I merged 102 Tiff files downloaded from ASTER: https://earthexplorer.usgs.gov/.

They were each 17 MB, plus or minus. I merged them using the Processing toolbox/GDAL merge tool. I am trying to use the raster calculator on it, but I get an error message saying Insufficient memory available for the operation. The pixel type is float 32, and I heard at my previous question that this may be making the file size too big.

About my computer:
7.8 GiB of memory
32 bit
483.8 GB disk space (128.5 used)
Ubuntu 16.04 LTS
QGIS 2.18.11

I have tried changing the pixel type with GDAL Translate, but I get an error message saying no output file was created:

When I use r.mapcalc to do the raster calculation, I click run and it seems to work, but no file is saved and no layer appears. I tried using both the name of the file in the Layers panel and its filename.

I could try merging the different rasters into small groups, but idoes anyone know an easier way?

Answer 1

Well, judging by the first screenshot you provided, seems to me that you are not specifying the output file to produce to the translate, which you should:

This is probably the reason why you get the "no output file created" warning.

Besides that, I don't see why you could be running out of memory from a rastercalc. Seems that 102*17MB is 1734MB, or about 1.7GB. Personally, I have run raster calculations (NDVI, etc.) on Tiff files of about 3GB without problems at all; sure it's takes some time but it outputs the calculation.

Some other things you could check are:

• Make sure the output folder for your calculation has write permissions.
• Check if that disk space is really what you have available, perhaps your partition or similar has less space available.
• You can also try doing what suggested here, and use r.mapcalculator or well gdal_calc

Alternatively, you can consider loading your Tiff on Python and doing the calculation on your bands there. The way I do it is to open the file with OpenCV (or with imread for large images) and then operate it with a Numpy array, something like:

from skimage.io import imread
import numpy as np
#read image
img = imread("yourImage.tif")
#load as float 32 Numpy array
img = np.float32(img)

#then do your calculations, for example NDVI:
#(B1-B3)/(B1+B3)
#Using Python's slicing, operate on all pixels by bands
ndvi = (img[:,:,0] - img[:,:,2]) / (img[:,:,0] + img[:,:,2])

You can then use your calculation ndvi as you wish.