How do I cluster XY point data? (ArcGIS 9.3)

Question

I have a whole bunch of icons displaying on a map, I want these clustered together as was suggested to me for a different question here... ArcGIS 9.3. How do I increase Picture Marker Symbol performance?

A link to an example was provided but I'm not really sure where to begin. I'll preface this by saying I'm a programmer way before I'm a gis person.

What I want to know, in total noob terms, is how can I cluster my icons for XY events on the map?

Thanks!

EDIT: Forgot to mention, this is for a C# web ADF GIS portal... If it makes any difference.

Is this even possible? I can't seem to find anything for doing this with 9.3 adf.

Answer 1

A fairly simple way is to snap the events to a grid. It's fast enough that potentially you can do it dynamically.

You can snap the points by means of a few simple computations before creating the events. Decide on the grid's origin and mesh size, using the same coordinate system as the (X, Y) values you have. Let the origin have coordinates (Ox, Oy) and suppose its mesh is c (the distance between neighboring points along the grid's basic directions). Large values of c will create large clusters, at the cost of potentially moving some points large distances (up to c*Sqrt(1/2)). Exploit the flexibility to choose c by finding a value that accomplishes the tradeoff you want between positional accuracy on the map and simplicity.

Calculate

I = {(X - Ox)/c}

J = {(Y - Oy)/c}

Then compute

U = I * c + Ox

V = J * c + Oy.

The braces "{}" mean "round to the nearest whole number." (You could do these calculations in two steps, rather than four, but we will see below that I and J have additional uses.)

After completing the calculation, designate (U, V) as the event coordinates instead of (X, Y). That takes care of the clustering.

In addition you want to winnow the records so there is a single point marker for each unique value of (U, V). Accomplish this by summarizing on (U, V) and basing the point events on that summary. I recall that ArcGIS is limited to summaries on a single physical field. You can create a field corresponding to (U, V) for this purpose with a computation like

Id = I + 10000*J

The value of 10000 only needs to be larger than the total number of (invisible) rows covering your data in the grid.

Answer 2

Since I saw your question yesterday, I was looking for an old notepad with a workflow I used to perform this clustering. It was for a fairly static layer, so I did not need to perform it on the fly, but I don't see why you couldn't wrap the workflow in a model builder process and use it as a geoprocessing service. It is based on out of the box tools of the Arc Toolbox:

• In Data Management, Feature Class, use Integrate on your point layer. With an appropriate threshold, it moves points close enough to the same spot, without clustering them;
• In Spatial Statistics, Utilities, use Collect Events on the resulting point layer to cluster points at the same position and count them;
• If needed, in Analysis, run a Spatial Join between the output and the input of the Collect Events to perfom calculation on attributes;
• If needed, clean results by dropping unnecessary fields.

If someone has a cleaner, simpler solution, I'd be interested. This is the one I used and it worked for me.

Answer 3

Since you're a programmer, you won't mind psuedocode...

First though: I'm not a big fan of clustering data without good reason. It hides information. Better to use a faster draw / smaller icons if you can. :)

K-means clustering is really simple to implement yourself:

create k cluster centers by picking k random points from your collection of points.
while not done:
   assign each point to the closest cluster center
   update the cluster centers to be the average of all points in that cluster
   done = true if no points changed cluster.

(There are faster algorithms, but they're more complicated. And chances are this is fast enough!)

Answer 4

You mention this is for a C# WebADF app - Not seen a way to do this, and imagine it would be pretty tricky.

However, the WebADF is being deprecated, so perhaps think about migrating your app to one of the ESRI APIs. :)

Mansour's (ESRI Flex Guru) blog, has an excellent post on how to do this with the ESRI Flex API, including a link to the source code.

• See the live demo here.

He also refers to how this can be done with Google Maps, utilising Maptimize.

He also has several other posts on how this can be achieved with some ActionScript/ESRI Flex API.

• Weighted Clustering - demo here
• Flaring Clustered Map Points - demo here
• Map Points Cubic Clustering - demo here

Answer 5

By simplifying or generalizing the data you can make a layer for each zoom level. You could calculate a vaule to (sum) or average for the relevant fields for display or query purposes.

Each layer would then be set for it's relative view range scale. Are you familiar with how to do this in your software?

New info:
Using the process described in GuillaumeC's thread to come up with several "shapefiles" each grouping the points together tighter.

In Arcmap you decide the outer and inner zoom scale that your user will use to view the points. Depending upon how many layers you broke your data into you correspond the scales to those files. Right click on the layer and in the general tab there is "do not show out beyond" and "do not show in beyond".

Plug in a top and bottom for each layer (e.g. 0-1000, 1001-1499). Combine them into a group, turn them all on and turn on the group. As you zoom to each breakpoint you will see the layers turn on and off. You can make a tooltip for the value that you calculate as described by GuillaumeC that will allow the value to show when th euser hovers on the points.

Answer 6

Have a look at this plugin maybe help you (R Point Clustering).

http://arcscripts.esri.com/details.asp?dbid=16444