I am not 100% sure if GIS is the correct stackexchange for this, but here we go.
A problem I have been bested by: I have a raster layer of approximately 240 million cells. It could be a DEM or anything similar. I also have a series of sampling-site polygons of irregular shape located throughout the extent of the DEM but only occupy about 0.1% of the space. I am using statistical tests to compare the distribution of values (elevations in this example) between the sample-site locations and the overall DEM (considered background values).
I have used the MASK() function in the Raster package of R to extract the the DEM values where they intersect non-Null values of the sample-site raster. It returns a raster that is 99% NA and the remainder are the values at the sample sites.
From here, I draw roughly 50K random samples from the DEM and I test them against the distribution of values at sampling-site locations. However, I have only found the most inefficient ways to remove or ignore the NA values (eg.
x <- na.omit(y))
*edited to add: What is an efficient way to extract the values from the sampling-site raster into a vector that contains no NA values?
I aware of the
na.rm flag fro some functions in the raster package, but I have not used it to success.
I have successfully implemented this general process by turning the sampling-site raster into cell centroids within sampling-sites and extracting values, but that has extra steps. I apologize for being so verbose, but I would appreciate any and all ideas.
edit* this is the most effective way I have found is
site_smpl_freq <- as.data.frame(freq(site_smpl, useNA='no', progress='text') but it still takes a few minutes to run
edit* to add a code example.
###sample background data and put in data frame rand_smpl <- sampleRandom(raster("DEM"), collapse =''))),50000) rand_smpl_freq <- as.data.frame(count(rand_smpl)) ### then a bit that modifies the table if values exceed a certain threshold ### extract raster values at the location of sampling-sites sites_extract <- mask(DEM, sampling_sites_raster) ### Here is where I create the freq table using the useNA option ### It works, but it takes a long time and seems inefficient site_smpl_freq <- as.data.frame(freq(sites_extract, useNA='no', progress='text')) ### then I modify the table as above if values exceed a certain threshold ### This is followed by some manipulations that add columns for cumulative counts and percentages, then to ggplot()