I am trying to do a point-in-poly analysis using R. I have 4.5 million points and 5k polygons. This is computationally intensive. I was pointed to parallel processing through another post and tried to run point.in.poly through parallel processing.

Without parallel

table1 = point.in.poly(spdf, all_polygons)

With parallel

library(foreach)
library(doParallel)

nrow = nrow(spdf)
registerDoParallel(makeCluster(no_cores - 1))
ptm <- proc.time()
foreach(i=1:nrow, .combine = rbind)  %dopar%  {
  table1 = point.in.poly(spdf, all_polygons)
  getValues(table1)
}
proc.time() - ptm
endCluster()

Without parallel results in about 7 minute processing time for a 100k subset. With parallel takes longer, then freezes and crashed my computer. Note that I am on a four core computer.

How might I speed up this function? Without parallel processing, my rough estimation is that this process could take 6 hours.

I am trying to do a point-in-poly analysis using R. I have 4.5 million points and 5k polygons. This is computationally intensive. I was pointed to parallel processing through another post and tried to run point.in.poly through parallel processing.

Without parallel

table1 = point.in.poly(spdf, all_polygons)

With parallel

library(foreach)
library(doParallel)
    
    
registerDoParallel(makeCluster(2))
ptm <- proc.time()
foreach(i=1:length(split_df), .combine = rbind, .packages='spatialEco','sp')  %dopar%  {
  myvars = c("LONG_X","LAT_Y")
  producers_lat_long = split_df[i][myvars]
  spdf <- SpatialPointsDataFrame(coords = producers_lat_long, data = split_df,
                               proj4string = CRS("+proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0"))
  clip1 = point.in.poly(spdf, all_polygons)
  getValues(clip1)
}
proc.time() - ptm
endCluster()

Without parallel results in about 7 minute processing time for a 100k subset. With parallel takes longer, then freezes and crashed my computer. Note that I am on a four core computer.

How might I speed up this function? Without parallel processing, my rough estimation is that this process could take 6 hours.

I am trying to do a point-in-poly analysis using R. I have 4.5 million points and 5k polygons. This is computationally intensive. I was pointed to parallel processing through another post and tried to run point.in.poly through parallel processing.

Without parallel

table1 = point.in.poly(spdf, all_polygons)

With parallel

library(foreach)
library(doParallel)

nrow = nrow(spdf)
registerDoParallel(makeCluster(no_cores - 1))
ptm <- proc.time()
foreach(i=1:nrow, .combine = rbind)  %dopar%  {
  table1 = point.in.poly(spdf, all_polygons)
  getValues(table1)
}
proc.time() - ptm
endCluster()

Without parallel results in about 7 minute processing time. With parallel takes longer, then freezes and crashed my computer. Note that I am on a four core computer.

How might I speed up this function? Without parallel processing, my rough estimation is that this process could take 6 hours.

I am trying to do a point-in-poly analysis using R. I have 4.5 million points and 5k polygons. This is computationally intensive. I was pointed to parallel processing through another post and tried to run point.in.poly through parallel processing.

Without parallel

table1 = point.in.poly(spdf, all_polygons)

With parallel

library(foreach)
library(doParallel)

nrow = nrow(spdf)
registerDoParallel(makeCluster(no_cores - 1))
ptm <- proc.time()
foreach(i=1:nrow, .combine = rbind)  %dopar%  {
  table1 = point.in.poly(spdf, all_polygons)
  getValues(table1)
}
proc.time() - ptm
endCluster()

Without parallel results in about 7 minute processing time for a 100k subset. With parallel takes longer, then freezes and crashed my computer. Note that I am on a four core computer.

How might I speed up this function? Without parallel processing, my rough estimation is that this process could take 6 hours.