How to identify outliers (not considering spatial relationship) using ArcGIS Pro?

Question

I am new to geospatial data and have a task to identify outliers from polygons. I have been looking at the different tools that ArcGIS Pro has to offer: Optimize Outlier Analysis, Optimize HotSpot Analysis, and Clusters and Outliers Analysis. Nevertheless, some of these tools required an input distance (spatial relationship).

I have a .shp file with about 400 polygons. Each polygon has columns in the attribute tables for 10 different dates and each date has a float value (range: 1-100). Example:

Column1 = 04/23/2021: 22.04
Column1 = 04/23/2021: 21.16
Column1 = 04/23/2021: 15.78

Column2 = 05/23/2021: 11.24
Column2 = 05/23/2021: 31.17
Column2 = 05/23/2021: 85.48

I would like to identify outliers for each date column.

Which tool should I use to find these outliers? I do not need spatial relationship to be considered.

Update: I tried the Multivariate Clustering tool and set the number of clusters to 6. These tool does not require input distance values. Do you think this would be correct to use for the task? From this tool, I can see the Features Per Cluster and identify the cluster with the lower Count (outliers).

Answer 1

Clustering is not a valid way to identify outliers! There are too many potential cluster solutions and the way the cluster centers are derived can have a profound effect in a statistic like k-means (which is under the hood of multivariate clustering tool). You can never be sure that a given cluster solution are actually capturing the clusters in the data, especially if there are more than one. ArcPro is not the place to perform statistical analysis, which is exactly what you are trying to do here.

For identifying outliers on the tails of the distribution I would recommend using something like the modified Z-score (Iglewicz & Hoaglin 1993) or in this case, potentially treating the data as a time-series.

Please, just keep in mind that outliers, due to tail distributions (fig 1), are very different than outliers within a time-series, which are extreme local deviations within the serial component of the data (fig 2). This is why time-series data is often smoothed before analysis. If you are after the time-series component, smoothing is the way to go else-wise you are getting into extreme number theory and anomaly detection, which is not trivial. You can smooth a time-series using a lowess regression, spline or Savitzky-Golay convolution filter.

A modified Z-score will only identify outliers due to extreme values at the tails. So, it is important to understand what type of outlier effect you are after and it looks like your data may have a serial component. Here is a simple expression of the modified z-score ( (0.6745 * (x - median(x))) / mad(x) ) where; median is the 50th percentile and mad is the median absolute deviation from the median. Should be easy to implement in Excel.

fig 1.

fig 2.

Answer 2

I know ArcMap has a tool called Grouping Analysis but reading the help file in ArcPro this seems to have been split out into 2 tools. If I have understood it correctly the Multivariate Clustering will group data based upon selected attributes only and not proximity in the geometry.