I need a Spatial Grid as a master grid for diverse thematic maps. How do I produce a Spatial Grid from a raster discarding all NA pixels?
You can get all the non-NA coordinates of cells in a raster with:
those are the cells that are not NA. You can then probably feed these to SpatialPixels
Although personally anything on a grid I'd keep as a raster.
I'm not totally sure still what it is you want -
To transform a RasterLayer to a Spatial* object (Grid or Pixels) you can use the coercion function "as"
Your two requirements seem to be about different things:
1) Some kind of reliable raster grid template.
2) A sparse grid that doesn't explicitly store missing cells.
sp::GridTopology provides the first one, it's just a description of the grid based on the lower left cell coordinate (cellcentre.offset), the cell spacing (cellsize), and the dimensions of the grid (cells.dim).
The sp::SpatialPixelsDataFrame class allows you to store sparse grids, but on its own it stores much more than the "template" - it also stores every coordinate explicitly. This is because it does two jobs, one it allows you to preserve the original coordinates that come from the grid and are possibly slightly irregular, two it lets you store only those cells that have valid values. (Arguably* these two goals should have been separated, but that's another story).
I don't think the raster package has an explicit analogue to GridTopology, but you can get hold of the components to "roll your own":
Linking these all together, we can go from raster to sp:
(Note how the dimensions have to be reversed). Another simpler way is to coerce to SpatialGrid and use sp's getGridTopology, though this is more expensive since the coordinates end up being generated along the way:
Those three parts of the raster "topology" are not all necessary, since some are redundant but there isn't otherwise a way to capture them all as one object - except, that raster created above is "empty" and so it can do the job that GridTopology does for sp. I'm not sure on the details of how "empty" it is, but it certainly does not explicitly store the "data" slot and is smaller than it would be with values in it. The raster package does in general do its best to keep memory usage to a minimum, and so with it you might not need to worry about really being "sparse".
That might help explain a little more, I know I'm overlapping Spacedman's answer but it's still not clear exactly what you mean in the question yet.