Getting similar IDW results using Spatial and Geostatistical Analysts for spatial interpolation of data with many zeros?

Question

My data are plant biomass data from hundreds of sites. Most sites have zero biomass; these zeros are real. Thus, my data violate most assumptions of geostatistical interpolation methods (confirmed using Geostat. toolbox): non-normal distribution without log transformation (zero problem), clustering of sites with biomass, data are mostly stationary except near transitions out of vegetated areas, autocorrelated over very short distances.

I conducted IDW and Local Polynomial interpolations using Geostatistical Analyst and received mediocre results. The main problem is that there are very few areas predicted to have zero biomass.

In Spatial Analyst, simply selecting a small number of points with a variable search tracked the data much better. How can I get similar results using IDW in Geostatistical Analyst in order to get at least one measure of fit (RSE)?

Can my process/decision for interpolation be adequately justified for a paper, or should I be doing something else?

Answer 1

Blockquote Can my process/decision for interpolation be adequately justified for a paper, or should I be doing something else?

Well, +1 for this one.

Bear with me, I am not a (geo-)statistician at all, but I am always a bit stumped when I see people trying to interpolate datasets that simply aren't suitable for interpolation, even in the face of exploratory spatial data analysis results clearly showing they shouldn't be interpolating the data.

It is probably true that nowadays there are better interpolation tools and methods than maybe 20 years ago when I studied, that can also better deal with more "erratic" data, but people often seem almost blindly to assume that interpolation is the only way to deal with any dispersed "point" datasets, and that if they can derive a surface, it is "the truth".

Even worse, people often fail to see the great opportunities their datasets may offer for "normal" (non-spatial and geostatistical) statistical analysis. They may have measured a plethora of factors possibly influencing their depended variable, but proceed to just interpolating their main variable of interest.

Having studied biology too, I know from experience vegetation data varies very non-continuous. That type of data generally is not very suitable for any kind of geostatistical interpolation, contrary to stuff like groundwater levels that tend to vary much more smoothly and continuous.

I would really start out by comparing your data with other environmental or geographic data and abiotic factors, like information about soil types, groundwater types and levels, sun and climatic exposure, incline etc. that may be real factors in determining biomass. Input those into a statistical package, and see if you can determine any statistical correlations with the biomass. If these statistical results aren't "enough" by themselves, you might possibly use those to classify maps of the existing abiotic factors / data to show areas that are likely to have low or high biomass potential. This wouldn't require an interpolation, your statistical analysis would be your guide.

Answer 2

You have a classic zero inflation problem and this data is just not suitable for interpolation statistics. You may want to try a regression approach using a zero inflated model (ZIP), where the zeros are modeled independently as a binomial process. Commonly, the non-zero model is a Poisson regression but, that is not set in stone and could be any distribution that is appropriate for your data. You could then just apply the coefficients to predict a raster surface. The independent variables could be as simple as [X,Y] coordinates and perhaps an autocovariance term to represent the spatial lag.

Additional options would be a non-parametric regression approach such as random forests or gradient boosting or k-nearest neighbor imputation using a classical method such as canonical regression. There are suitable implementation of all of the methods mentioned in R as well as spatial classes that will handle the data and predictions. Sorry, but to solve this problem, you are now outside of the ESRI ecosystem.