Python 2.7.4 32-bit
GDAL 1.10.1
I have a Python script which takes a point file with x,y,z coordinates, makes a vrt of it and then tries to create a Geotiff from it with gdal_grid. During testing all goes well and then I try to put in a 3.5 GB file. It fails. I try a junction of that file and it works but at a certain moment when my test file gets to big it fails again. The error is always the same:
ERROR 1: CPLMalloc(): Out of memory allocating 64 bytes.
FATAL: CPLMalloc(): Out of memory allocating 64 bytes.
ERROR 2: Cannot allocate 6563 bytes
The last amount of bytes is equal to the width of the geotiff it should create. The fact that it works with smaller files makes me think I'm running into restrictions of the software.
If that is the case I would like to know if there is a workaround?
gdal_grid -outsize 50000 50000 points.shp -l points foo.tif
Play with outsize and you will see if gdal_grid is setting some limits. If you can exceed the size where you experienced troubles then the fault is either in Python or in your code. Or perhaps you meet the limit of tiff and gdal_grid does not change the switch into BigTIFF automatically. In that case you may need to use-co BIGTIFF=YES