I want to create a weight matrix based on distance. My code for the moment looks as follows and functions for a smaller sample of the data. However, with the large dataset (569424 individuals in 24077 locations) it doesn't go through. The problem arise at the nb2blocknb fuction. So my question would be: How can I optimize my code for large datasets?
# load all survey data DHS <- read.csv("Daten/final.csv") attach(DHS) # define coordinates matrix coormat <- cbind(DHS$location, DHS$lon_s, DHS$lat_s) coorm <- cbind(DHS$lon_s, DHS$lat_s) colnames(coormat) <- c("location", "lon_s", "lat_s") coo <- cbind(unique(coormat)) c <- as.data.frame(coo) coor <- cbind(c$lon_s, c$lat_s) # get a list with beneighbored locations thath are inbetween 50 km distance neighbor <- dnearneigh(coor, d1 = 0, d2 = 50, row.names=c$location, longlat=TRUE, bound=c("GE", "LE")) # get neighborhood list on individual level nb <- nb2blocknb(neighbor, as.character(DHS$location))) # weight matrix in list format nbweights.lw <- nb2listw(nb, style="B", zero.policy=TRUE)