3

I am trying to create a near table to link two datasets. The first consists of the outlines of about 1.8 million buildings, the second a 3 meter buffer around these buildings, which as been merged together where multiple buffers overlap.

I want to identify the buffer closest to each building. I used the Generate Near Table tool which appears to be working, but is running very slowly. I left it running for 47 hours and it only managed to produce 4700 records. As I have 1.8 million to do I want to know if there is anything that I can do to make this tool run more efficiently.

I've got the 64 bit geo-processing installed and enabled multi-core processing.

I don't think it is the computer which is 6 core @ 3.2 GHz, Windows 7 64 bit, 16 GB of ram, with Solid State Disk.

Improve this question
7
  • 1
    I think it's a problem with how the tool works...at a guess, it appears to be using the underlying database to do the spatial queries. Doing a similar task, with 2 million points near ~600,000 lines (buffered 330 feet) takes over 5 days on my machines. – Evil Genius Nov 29 '13 at 15:08
  • This is a pretty ugly query, and 1.8m iterations of it doesn't make it prettier. What format are you using for data storage? How spatially fragmented is the data? – Vince Nov 29 '13 at 21:02
  • You may be better off buffering without the dissolve, obtaining a 1-to-1 correspondence, then dissolving the buffers and using the Identity tool to determine which individual buffers are contained in which dissolved buffer – nmpeterson Nov 29 '13 at 21:03
  • 1
    Make sure your data have spatial indices, easy way to ensure that is to have your data in a file geodatabase. Is there any way to subset the data? For example you are never going to do this analysis across county boundaries so split the data up into counties. Also you say each building has a 3m buffer and you want to find the nearest buffer. For the non-overlapping buffers surely there is only ever 1 buffer? These isolated building/buffer combinations could be dropped from the analysis? – Hornbydd Nov 30 '13 at 1:11
  • The 10.2.1 will have significantly improved performance of the generate near table for polygons and polylines if it helps you. It also will have an option to use geodesic distance. – 0kcats Dec 10 '13 at 12:12
1

As mentioned by @0kcats in the Comments, the ArcGIS 10.2.1 for Desktop release may well have addressed your performance concerns:

The Generate Near Table and Near tools have been completely rewritten to be dramatically faster and they now have an optional Method parameter that determines how distances are computed. When Method is set to GEODESIC, distances are calculated across the earth's surface. This is most accurate when the distance between features is large and you want to minimize the distortion Inherent in all projected coordinate systems, particularly in projections like Web Mercator. When PLANAR, Euclidean distances are calculated using the coordinates of the features and is appropriate for projections that minimize distance distortion or when the distance between features is small.

The improvements made to Generate Near Table have not yet been implemented for the Point Distance tool. In most workflows, you can use Generate Near Table in place of Point Distance.

Improve this answer
0

1) Maybe try setting the Parallel processing cores setting to 100%.

http://resources.arcgis.com/en/help/main/10.2/index.html#//001w0000004m000000

type 100% in the environment setting and I think this tool honors the setting so on your PC you will dedicating 6 cores to the process as opposed to 1 or even half of one.

2) can you changed the merged buffers to point centroids (within) as well to make it point-to-point. This is computationally far more efficient but I understand it may not be possible.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.