Parallelize Extracting Large Raster Values (one band) to Points

Question

I have sf data with single point geometries (points) and a 23,000 x 18,000 elevation RasterLayer with a single band, loaded from a geotiff (elev). They have the same CRS. I want to extract the elevation values to the points.

When I try running extract(elev, points), since my raster is very large I get:

Error: cannot allocate vector of size 3.1 Gb

I followed advice here R - multicore approach to extract raster values using spatial points and tried to parallelize, but I only have a single raster, so chances are the work won't get divided. Still I try the following:

library(snowfall)
sfInit(parallel=TRUE, cpus=parallel:::detectCores()-1)
sfLibrary(raster)
sfLibrary(sp)

# Run parallelized 'extract' function and stop cluster
e.df <- sfSapply(elev, raster::extract, y=points)
sfStop()

I get the same result:

Error: cannot allocate vector of size 3.1 Gb

All the examples I've found so far, such as the above, store multiple rasters in a list. How do I parallelize extraction from a single raster like this? Or is there a way to make use of sparse matrices and memory management as noted here?

I'm on Windows 10 running RStudio 1.1.419 (64 bit). 8GB RAM. Intel Core i5-6200U CPU @ 2.3GHz 2.4GHz

Answer 1

You can split your image to tiles like 100*100 tiles, then loop tiles to extract points. Or you can try database solution with building spatial index. When you load your image to spatial database, it also can tile your image. PostGIS is open source, in PostGIS just a SQL query can do this job, the function is ST_Value.

[EDIT]You can loop records(here are tiles) using PL/pgSQL like below:

CREATE OR REPLACE FUNCTION work_on_tiles() AS
$BODY$
BEGIN
    FOR tile IN
        SELECT rast FROM public.elev WHERE rid > 0
    LOOP
        -- can do some processing here
        var = ST_Value(rast)
    END LOOP;
    RETURN;
END
$BODY$
LANGUAGE plpgsql;

SELECT * FROM work_on_tiles();