# ArcGIS: table statistics range, method?

I have the following table with distance values (NEAR_DIST) after using the "Near" tool in ArcGIS: I am looking for a method of getting the following statistical information/statement, for example: "80% of the values from the field "NEAR_DIST" fall inside the range of 40-500, 90% in the range of 30-600,..."

With the normal statistical information i do not get much information, just the sum, min, etc...: Is there a tool in ArcGIS to achieve that or how would you deal with that?

I would like to remove the "outliers" (for example the 868 and 528 (which are too high, and also the very low records), to get a better frequency distribution (histogramm)...therefore i would like to automatically find the "break values" which i can use to filter my table.

Is there a way in ArcGIS to do this, or any statistically method?

Here is a python solution, using arcpy to access the data and numpy to calculate the statistical values.

Using arcpy.da.SearchCursor() write the values to a list. Use python.numpy.percentile() to find the threshold percentile values that you want to use to identify outliers, lets take your example and drop the lowest 10% and highest 10% of values.

If you actually want to remove the records, include the last block of code, otherwise the print statement may be enough.

``````import arcpy, numpy as np

# input shape file or feature class
in_shp = r'd:\near_features.shp'

# write distance vals to list
vals = []
with arcpy.da.SearchCursor(in_shp, ['NEAR_DIST']) as cur:
for row in cur:
vals.append(cur)

# identify 10th and 90th percentiles (leaving 80% of values remaining)
percentile_10 = np.percentile(vals, 10)
percentile_90 = np.percentile(vals, 90)

# print percentiles
print '80% of records are between {} and {} distance'.format(percentile_10, percentile_90)

# delete records that exceed these threshold values
with arcpy.da.UpdateCursor(in_shp, ['NEAR_DIST']) as cur:
for row in cur:
if row <= percentile_10:
cur.deleteRow()
elif row >= percentile_90:
cur.deleteRow()
``````
• +1, great use of `numpy`. Instead of building a list and then appending each record to it, you can list a list comprehension to build it all in 1 go: `vals = [row for row in arcpy.da.SearchCursor(in_shp, ['NEAR_DIST'])]` – Paul Jun 23 '16 at 15:44

If you are looking for an automated way to do this, or just a way to do this at all, you could:

1. Perform a query on your table WHERE: NEAR_DIST >= 40 AND NEAR_DIST <=500

2. Divide how many records are returned (e.g. 800) by how many total records you have (e.g. 1000). For example 800/1000 * 100 = 80%

If you want to automate this you could do this in ModelBuilder. Just perform each query,count results, do the math and write it to a text file.

• Thank you for your feedback. For a better understanding: I would like to remove the "outliers" (for example the 868 and 528 (which are too high, and also the very low records), to get a better frequency distribution (histogramm)...therefore i would like to automatically find the "break values" which i can use to filter my table... Is there a way in ArcGIS to do this, or any statistically method? – Duddel Jun 23 '16 at 7:26
• I see in the edits that you added the outlier issue to your post. Looks like somebody has an automated solution so I'll just refer you to that :-) – alexGIS Jun 24 '16 at 18:06