# How to fully load a raster into a numpy array?

I have been trying to check my filters on DEM raster for pattern recognition and it is always resulting in missing last rows and columns(like..20). I have tried with PIL library, image load. Then with numpy. The output is the same.

I thought, something is wrong with my loops, when checking values in array (just picking pixels with Identification in ArcCatalog) I realized that pixel values were not loaded into an array.

So, just simply opening, puting into array and saving the image from array:

a=numpy.array(Image.open(inraster)) #raster is .tif Float32, size 561x253
newIm=Image.new(Im.mode, Im.size)
Image.fromarray(a).save(outraster)


Results in cuting away the last rows and columns. Sorry, can#t post the image

Anyone could help to understand why? And advise some solution?

EDIT:

So, I succeeded loading small rasters into numpy array with a help of guys, but when having a bigger image I start getting errors. I suppose it's about the limits of numpy array, and so array is automatically reshaped or smth like that... So ex:

Traceback (most recent call last):
File "<pyshell#36>", line 1, in <module>
File "C:\Python25\lib\site-packages\osgeo\gdal.py", line 835, in ReadAsArray
buf_xsize, buf_ysize, buf_obj )
File "C:\Python25\lib\site-packages\osgeo\gdal_array.py", line 140, in BandReadAsArray
ar = numpy.reshape(ar, [buf_ysize,buf_xsize])
File "C:\Python25\lib\site-packages\numpy\core\fromnumeric.py", line 108, in reshape
return reshape(newshape, order=order)
ValueError: total size of new array must be unchanged


The point is I don't want to read block by block as I need filtering, several times with different filters, different sizes.. Is there any work around or I must learn rading by blocks :O

-

if you have python-gdal bindings:

import numpy as np
from osgeo import gdal
ds = gdal.Open("mypic.tif")


And you're done:

myarray.shape
(2610,4583)
myarray.size
11961630
myarray
array([[        nan,         nan,         nan, ...,  0.38068664,
0.37952521,  0.14506227],
[        nan,         nan,         nan, ...,  0.39791253,
nan,         nan],
[        nan,         nan,         nan, ...,         nan,
nan,         nan],
...,
[ 0.33243281,  0.33221543,  0.33273876, ...,         nan,
nan,         nan],
[ 0.33308044,  0.3337177 ,  0.33416209, ...,         nan,
nan,         nan],
[ 0.09213851,  0.09242494,  0.09267616, ...,         nan,
nan,         nan]], dtype=float32)

-
Yeah, with gdal I guess I did not have problem, but I'm trying to use as less libraries... And numpy seemed so popular for that 'while googling'. Any idea, indeed, why numpy/PIL stops loading??? – najuste Sep 10 '12 at 7:51
I don't know. PIL should robust enough so its shipped with python. But imho geotiff are more than images - they carry lots of metadata for example- and PIL is not (again imho) the right tool. – nickves Sep 10 '12 at 11:37
I just sometimes hate those diff quotation and slash requirements, when opening data.. But what about writing numpy array back to Raster? It works with PIL library, but using outputRaster.GetRasterBand(1).WriteArray(myarray) produces invalid raster.. – najuste Sep 10 '12 at 21:44
don't forget to flush the data to the disk, with outBand.FlushCache() . You can find some tutorials here: gis.usu.edu/~chrisg/python/2009 – nickves Sep 10 '12 at 21:53
Check " lists.osgeo.org/pipermail/gdal-dev/2010-January/023309.html " - it seems you've ran out or ram. – nickves Sep 23 '12 at 16:40

You can use rasterio to read/write NumPy arrays. For example, read, edit a pixel, then save it back.

import rasterio
with rasterio.open('/path/to/raster.tif', 'r+') as r:
arr = r.read()  # read all raster values
print(arr.shape)  # this is a numpy array, with dimensions [band, row, col]
arr[0, 10, 20] = 3  # change a pixel value on band 1, row 11, column 21
r.write(arr)


This will read everything into a 3D numpy array arr, with dimensions [band, row, col]. As with any "with" statement in Python, the raster will be written and closed at the end of the with-block.

-

Granted I'm reading a plain old png image, but this works using scipy (imsave uses PIL though):

>>> import scipy
>>> import numpy
>>> img.shape
(81, 90, 4)
>>> array = numpy.array(img)
>>> len(array)
81

@najuste What OS are on? Mac and most Linux flavors come with scipy and numpy. – Chad Cooper Sep 10 '12 at 22:18