1

I'm trying to do a very simple analysis where I need to find all land cover of a certain type within 30 meters of a stream. I have a raster from the National Land Cover Database (NLCD) from which I've pulled out the relevant pixels and the National Hydrography Dataset stream network.

All I need to do is buffer the streams by 30m and either:

  1. Vectorize my NLCD land cover pixels and clip by the stream buffers, or ...
  2. Rasterize my stream buffers and pull out the NLCD pixels that overlap them.

(I know these aren't quite equivalent methods in terms of results, but that's OK for the purpose of this question)

The issue is that I need to do this for the whole U.S. Even if I break up the streams and the land cover by county (i.e. into ~3,100 pieces) to avoid out-of-memory errors from buffering/clipping/etc. such large datasets, and it takes 1 minute to process each county (WAY optimistic, it seems to me), the analysis would still need to run for >50 hours!

Is there any way optimize this so I can get the runtime down to perhaps 24 hrs?

I am working in ArcGIS Desktop (ArcMap 10.3.1). My data is a line feature class in a file geodatabase (for the streams) and an 8-bit unsigned integer TIFF that I've imported into that same geodatabase. Everything is projected to USA Contiguous Albers Equal Area Conic.

Improve this question
8
  • 1
    If you have spatial analyst extension then neither 1 or 2 is a good option, use a tool like zonal statistics from a Con extracting the certain type of land cover you're interested in. I see that you realize you will need to break up the processing as your 30m buffer is only valid in a projected coordinate system. The runtime is directly linked to the cell size but projecting rasters takes longer than projecting vectors; if you can obtain your desired metric from the geographic data it could be faster to project the buffers instead of the rasters.
    – Michael Stimson
    Commented Oct 31, 2019 at 1:44
  • 2
    Tiling by county is highly wasteful. Use a regular fishnet (polygon) which aligns to pixel size and is 2-3 times taller than it is wide. You might consider building in overlap into the tiles (shifting the LL corner southeast by at least the buffer distance) so that each tile can be safely processed independently, then mosaicked afterward.
    – Vince
    Commented Oct 31, 2019 at 1:57
  • All my streams and land cover is projected to USA Contiguous Albers Equal Area Conic & I've already used Con to extract only the land cover pixels of interest. I'm not sure how zonal stats using buffers as the zones would be more workable than my option #2, since doesn't zonal stats rasterize zone polygons (the buffers) as an initial step anyway? Rasterizing these buffers seems to take unreasonably long in any case. As for fishnets, I've never used them before -- could you say more about how creating countless very small tiles is superior to using counties & how to use these tiles?
    – Evgeni
    Commented Oct 31, 2019 at 2:16
  • 2
    Not countless, no more than a few hundred (aligned to the raster, not replacing it). If you don't yet have an SSD, buy one; two orders of magnitude in I/O performance should bring this down to an hour or two.
    – Vince
    Commented Oct 31, 2019 at 2:59
  • By the time you've collected all the replies here, 50 hours will have passed and you'd have gotten your answer if you just used your brute force approach. And of course, the counties are independent, so use two computers and you'll be down to about 24 hours like you're asking for.
    – Llaves
    Commented Oct 31, 2019 at 3:51

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.