# How to handle interpolation of multiple (quality) tiers of irregular data?

I'm going to try to provide a high level overview of the issue - realizing that more details/info will be necessary - but I don't want to write a novel that seems too overwhelming for people to respond to :)

Background:

Let's say I have 3 sets (A, B, C) of irregular point data of varying quality; A is "better" than B and B is "better" than C. One or more data sets might have data for a given point.

Goal:

Create a grid/map that contains the "best" possible data.

Example/Possible Solution:

Use some sort of inverse distance weighted algorithm in combination w/somehow weighting the better datasets more than the lower quality data sets.

Please feel free to post follow-up questions as I know my original post is rather vague. Thanks in advance.

• Good question, what your asking is not trivial. I think this question is more appropriate for crossvalidated, the statistics stackexchange. Getting this done will probably require a tool more flexible than Arcgis, e.g. R or matlab. – Paul Hiemstra Dec 26 '11 at 16:47
• If using R is an option, you could also aks your question at the R-sig-geo mailing list. Do mention that you where refered there from stackexchange to avoid complaints about cross-posting. – Paul Hiemstra Dec 26 '11 at 18:03
• @Paul - Sorry for the delay. Agreed on the non-trivial part :) There is really no restriction on what the solution can be. I have no issue w/using R, matlab or even writing my own tool in C, fortran, or any other language. I'd obviously prefer FOSS but it is more important to get the right answer than to do things on the cheap. – malonso Dec 28 '11 at 12:13

When "best" is interpreted as "minimum variance unbiased estimator," then the answer is cokriging. This technique views the three datasets as realizations of three correlated spatial stochastic processes. It characterizes the process and their correlations by means of a preliminary analysis, "variography." The variography can be informed by additional considerations such as the relative quality of the datasets: this would cause the analyst to prefer variogram models that assign higher variances to the lower-quality datasets. Then, the co-kriging algorithm itself uses the data and the variographic output (a set of "pseudo co-variograms") to predict any desired linear combination of the three datasets (such as the mean values of dataset "A" within specified grid cells).

A simple, limiting version of co-kriging supposes the three datasets are not correlated with each other at all. In this case, each of the datasets can be kriged separately. (This means a variogram model is fit to each dataset separately and used separately to interpolate the data values onto a grid of specified locations.) The kriging output includes a kriging variance. Assuming each dataset represents the same physical quantity, the question becomes how to combine the three estimated values at each grid point into the "best" possible estimate of that quantity. The answer is to take a weighted average of the three estimates, using the reciprocals of the kriging variances as the weights. The result remains unbiased because the three inputs are unbiased by construction; weighting inversely by variance assures the result has the smallest variance, which is exactly what "best" was assumed to mean.

Kriging and co-kriging are available in the Geostatistical Analyst add-on for ArcGIS (at an extra cost) and in freely available software such as the gstat package for R. These are not activities to be undertaken casually: they are sophisticated, comprehensive analyses of the data that become valid only as a result of reasoned, accurate statistical characterization. Although software for kriging and co-kriging has long been available in many GIS environments, IMHO kriging is not an activity to be performed solely by a GIS analyst, because it requires close collaboration both with a seasoned statistician and an expert in the field of study. (Occasionally two of these three roles or even all three may be adequately filled by the same person, but this is rare.) It is one of those especially dangerous capabilities of a GIS because kriging output is easily created by anyone who can push the right buttons (a task that can be learned in 30 minutes with ESRI's excellent tutorial on Geostatistical Analyst, for instance) and will often look correct but is just the "GO" part of the proverbial GIGO.

• Fantastic response! This is out of my comfort zone so let me go over this a couple more times to make sure I grasp exactly what you are saying. Thanks again! – malonso Dec 28 '11 at 12:21

One thing I can come up with is to use simulation. For each of your observations you have a probability distribution, e.g. a normal distribution. For the low quality observations these probablity distributions are larger. From each of these observations you can draw a realization of the prob. distribution, interpolate the values, resulting in a map. Doing this a few thousand times will get you a probability distribution for each gridpoint in the inteprolation. From this you can extract for example the mean. In addition, you can use the probability distributions at the interpolation gridpoints to get an estimate of the uncertainty at each location. You probably cannot draw independent values at of your observations, but have to draw with a spatial dependency between the points.

I'm not sure how this would work from a statistical point of view using e.g. kriging, i.e. is the prob. distribution at the interpolation points valid given all the statistical assumptions in the realizations of the input points.

This kind of thing can probably most easily be done in a programming environment such as R or matlab.

• In simpler settings, simulation is often a good idea. However, in this case accurate simulation is not possible without some multivariate probability distribution to characterize the spatial correlations of the data. Otherwise, the result necessarily leads to exactly the same prediction at every point on the map. This makes simulation of spatial processes actually more difficult than prediction (kriging). – whuber Dec 27 '11 at 15:10