2

I am trying to count how many points, out of 17 million, fall inside a polygon (see image below) Polygon layer, in which I am counting the number of points that fall inside it

which I created from dissolving various buffers and features. The polygon layer is thus one dissolved feature. Both the points and the polygon are feature classes saved inside a file geodatabase. I am also using ArcGIS 10.3 on a 64 bit desktop.

To count the number of points I first added a column called 'Count'and populated all fields with 1. I then right clicked my polygon layer, clicked 'Join and Relate', and then 'Join'. I then chose the join option: 'Join data from another layer based on spatial location'. The parameters I entered are below:

Join parameters. As you can see, I am summing all fields. Thus, I will be able to sum the 'Count' column to determine how many point features lie inside the polygon feature as shown above.

I started this process 23 hours ago, and so far only half of the 17 million points are processed. I am also concurrently running two other similar processes with copies of the 17 million points and two separate polygon layers (both point/polygon feature classes are saved in separate geodatabases).

I was able to successfully do a similar join with the 17 million points and a fishnet with a grid size of 1 mile by 1 mile covering the state of Ohio. That process only took me 2.5 hours yesterday.

Why is it taking so long to sum all points inside a dissolved polygon feature class? Is running such a process 3 times at the same time adding to the slow speed? Is there a way to speed up this process in ArcGIS 10.3?

I am the only GIS Analyst at my company and am at a loss in understanding why it is taking so long.

Improve this question
11
  • 2
    How much RAM is available on your 64-bit PC? You might want to do tiled processing 'adaptive subdivision processing ' desktop.arcgis.com/en/arcmap/10.3/tools/supplement/… – Mapperz♦ Oct 6 '16 at 14:27
  • 3
    Your polygon layer is one feature and its BBOX covers the whole site and then spatial index does not help. All 17 million points must be tested against the complicated polygon ->slow. In your fishnet case each cell of fishnet is small, spatial index can be used for selecting a small amount of candidate points -> fast. Split your huge polygon into a bunch of smaller ones and you can make the spatial index to kick in. See gis.stackexchange.com/questions/212007/… – user30184 Oct 6 '16 at 14:45
  • 1
    Neither RAM, nor faster disk, nor PostgreSQL will address this problem; only splitting the massive polygon will help, and it will make two or three orders of magnitude of difference. THEN you can experiment with an in_memory workspace to increase performance. – Vince Oct 6 '16 at 22:50
  • 1
    It's now dated again, but this blog post describes the difference in mean point-in-polygon query response when large shapes are partitioned by a regular lattice. – Vince Oct 7 '16 at 1:29
  • 1
    Your fastest and most accurate option is to partition the large polygon with a latice into much smaller and less complex polygons that benefit from a spatial index and then do Select by Location on the points and use a summary or just open the point table view and look at the selected feature count to get the selected point count. This will give an accurate count and be much faster. You cannot do a Spatial Join using the partitioned polygons, since any points that touch more than one polygon boundary will be counted more than once when you sum the counts of all the polygons. – Richard Fairhurst Oct 7 '16 at 5:59
1

Your fastest and most accurate option is to partition the large polygon with a lattice into much smaller and less complex polygons that benefit from a spatial index and then do Select by Location on the points and use a summary or just open the point table view and look at the selected feature count to get the selected point count. This will give an accurate count and be much faster. You cannot do a Spatial Join using the partitioned polygons, since any points that touch more than one polygon boundary will be counted more than once when you sum the counts of all the polygons.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.