# Interpolating Maps - Statistical Learning Techniques vs Spatial Statistics Techniques

I have recently begun experimenting with the interpolation of variales over a whole country.

I have good sized training set where I have several dependant variables (continuous and categorical) associated to many independant variables. The independant variables are available for the whole country.

I'm not at all an expert in spatial statistics so my first approach was to train machine learning models like random forests, svm, gradient boosting, etc Which are behaving very well.

But I have been finding out that there are default methods for interpolating available in gis packages. Kriging. I played a bit with Kriging was not satisfied. Could co-kriging - regression-kriging get to better perfomances than these before mentioned methods (rf, svm)?

Also, I have noticed that there are methods which seem to be tailored for these problems like geographically weighted regression. I am just wondering if my approach is somewhat off if there are methods specifically designed for these tasks.

Thank you.

• One will never be satisfied just by "playing" with Kriging. It's a bit like saying you're new to rocket science and have been playing with rockets but just can't seem to get to the moon yet. Kriging is based on a random field model that takes experience to appreciate and work with; it requires good knowledge of the application domain and of statistics to use appropriately. Having said that, kriging has definite limitations, of which the most notable is its assumption of "stationarity," which restricts how much the interpolation can vary in nature across a spatial domain. – whuber Feb 8 '13 at 17:24
• What I meant by not satisfied is that using ordinary Kriging (for example) I would waste the information that I have in the independant variables I mentioned I have. Which lead to my second question - regression kriging? ... geographically wighted regression? Or ordinary Kriging can get to grand performances? From a brief overview I would think that it over smoothens the behavior of the inerpolated variable. – JEquihua Feb 8 '13 at 17:36
• (1) By means of "generalized linear spatial models," Kriging has been extended to accommodate covariates (as well as categorical data). See the geoRglm package. (2) Definitely kriging smooths the data: that's the nature of interpolation. If you want something else, you're probably looking for simulation: but that can also be conducted using kriging techniques, and is also supported in `geoRglm`. – whuber Feb 8 '13 at 18:42
• @whuber, I have been curious about spatial-GLM based kriging models. My spidey sense goes up a bit with the idea of using a spatial-GLM to specify a Kriging model. Don't you risk preconditioning spatial structure, where there is none, by using a model with an explicit spatial term? It seems like you have to have a pretty good idea of your spatial process before model specification. This one of the fatal flaws in genetic spatial assignment tests. – Jeffrey Evans Feb 8 '13 at 20:13
• @Jeffrey It depends on what "explicit spatial term" means. In kriging, location enters through specification of the covariance structure, which is estimated from the data. The spatial GLMs simultaneously estimate the covariance and the regression coefficients (using ML), which is better than traditional kriging which estimates the covariance (via variography) and then pretends the estimates are perfectly accurate (for the kriging step). Your concern, though, is a good one, and there is extensive literature discussing it. Many have found kriging to be a fairly robust approach when done right. – whuber Feb 8 '13 at 20:24

In your case, where you have a multivariate problem, ordinary Kriging is quite inappropriate. I find your interpretation of this as an "interpolation" problem is a bit off base as well. This is an estimation problem and more suited for Machine Learning or spatial regression, not geostatistics. The grey area are Splines. This can be a univariate interpolation method but can also be used in a semiparametric form to fit a multivariate nonlinear model and estimate a surface.

I will just say now, GWR is off the table. There are considerable problems with this method and it is really only suitable for exploratory analysis of nonstationarity. There are several papers that demonstrate the instability of GWR via simulations.

Regression Kriging is a promising method but it is very important that you read the primary literature to understand what it doing. If you distill regression Kriging to its basic form your are, in essence, Kriging the residuals of a linear model. The entire point is to violate iid assumptions so you have structure in your residual error. With complex high dimensional multivariate space this estimate may be difficult to interpret and plausibly, nonsense. It is really most suited for nicely parsimonious models.

Spatial regression and mixed effects methods will require you to fit a model. Since your problem is not inferential in nature this seem like overkill and is a steep learning curve.

As far as machine learning approaches, be aware most methods do not explicitly account for spatial process without help. There are proposed methods such as the inclusion of naive spatial process via [X,Y] coordinates, including a fit Nth order polynomial of [X,Y] or the specification a distance matrix as covariates.

Because you want to include both continuous and categorical covariates you are limited in methods. It sounds like your use of machine learning methods are the best suited to your problem.

It is critical that you clearly specify your problem and then select a statistical method to fit the problem. The approach throwing methods against the wall to see what sticks is quite unsatisfactory. It would be beneficial for you to review the literature in order to understand methods, model assumptions and implications of utilizing a given method. There is considerable information online that can be discovered with a simple Google Scholar search.

• Although I agree with much of this reply (+1), which reflects insight and experience, I think that kriging and geostatistics are unrealistically characterized. Specifically, I don't understand the distinctions you are maintaining between "interpolation" and "estimation," especially when kriging is actually a predictor, which seems to be exactly what the OP is looking for. As a matter of philosophy one might elect to use machine learning methods in an effort to create a useful predictor, but if the advantages of a probability model are desired, geostatistics is essential. – whuber Feb 8 '13 at 18:47
• " if the advantages of a probability model are desired, geostatistics is essential" (sic) Could you detail this a bit further please? – JEquihua Feb 8 '13 at 19:22
• @whuber, could you please elaborate on how you can specify a geostatistical model utilizing multivariate data? There are certainty cases where bivariate models apply and you can use geostatistical methods such as conditional simulation to quantify uncertainty in multivatiate model estimates but I just do not understand your point in relation to probabilistic outcomes. I am following the published statistical language regarding statistical model estimation and interpolation which may be semantics but is common nomenclature. – Jeffrey Evans Feb 8 '13 at 20:05
• Alas, I too am using standard terminology! See, for instance, the first half page at tinyurl.com/b32g6w5. For multivariate responses one uses cokriging; for multiple explanatory variables (in addition to location) one uses residual kriging or a spatial GLM. A singular feature of all the kriging technologies is their built-in methods to address change of support. A probability model enables this; it can assess interpolation error--no matter what interpolator is used; and it measures spatial correlation, which is useful in its own right. – whuber Feb 8 '13 at 20:20
• @whuber, that link to "change of support" goes to the home page of the Repositorio Academico of the Universidad de Chile, not to a specific article. Can you fix? I'd like to read it. – J Kelly Jan 1 '17 at 3:50

Geostatistics is a sorted of butchered pseudo-science, that restricts thought patterns to a dated simplified Gaussian process. It suffers from stationarity issues and fairly intense user/consultant intervention. These people generally defend this overused technique for obvious reason of self interest without any significant understanding of the larger world of estimation. Also the word statistics is incorrectly used as its association with pure mathematics is very limited.

You should be considering machine learning multi-kernel Gaussian processes that, by nature and definition, incorporate essential Bayesian statistical technology.