Does anyone know of a programme that will allow kernelling that takes a boundary into account in its calculation rather than simply masking areas that are impossible?

So far I have found: GME add on for Arc10 (Hawthorne Beyer) - I get an error everytime I specify a boundary shapefile. I've tried lots of different shapefile types and boundary complexities etc. It works fine when I don't specify a boundary.

AdehabitatHR package in R (Calange 2011) - this works well but the boundary you specify has to be very simple - line segments 3x kernel bandwidth in length, and not too tortuous. For my data this is a big oversimplification.

So I'm wondering if any other software can do it, GRASS or QGIS for example.


  • 3
    Recent paper with R code of potential interest, Border bias and weighted kernels. All the code is just ways to weight the observations to correct for border bias (if you can figure out the weights you can use whatever program to estimate the kde as long as it will except the weights). – Andy W Nov 5 '12 at 17:22
  • 1
    Precisely how do you want it to "account" for a boundary? There are many possible ways, ranging from masking to blocking the spreading to correcting for boundary effects. – whuber Nov 5 '12 at 17:37
  • 1
    Thank you @AndyW and whuber for the replies. I want to account for the boundary by preventing the kernel from spreading across it. The data are sea turtle locations, so I know that the utilisation distribution should not spread onto the land, but many of the locations are very near the coast so I don't want to bias against these locations by simply masking over the kernel parts that spread into the land. Thank you for the link to the paper - this looks great I will try to use similar code for my data. – KimS Nov 6 '12 at 13:27

At the core of kernel density estimation is the notion of distance. The best solution I know of is to use a better distance metric that accounts for boundaries, and varying costs of travel. It's best to choose a distance metric that fits the problem you're trying to solve. Overland terrain friction is great for hiking, but not for aerosol dispersion. Wind currents are essential for sailboat tracking, but irrelevant for driving directions.
Now that the notion of appropriate distance metrics is hopefully sufficiently motivated, I can recommend cost surfaces as a good general purpose distance metric. They are available in everything from ArcGIS, to R, to JavaScript and are fairly straightforward to construct. In qGIS for example, you can construct a raster friction surface and use that to calculate routes. Customize the friction surface to account for your boundaries, and you'll see the mass from the kernels around your points neatly spreading out around the obstacles.

  • +1 This is potentially a great approach. Having applied it in many problems, I have found that the principal difficulty is one not mentioned here: exactly how do you propose spreading the kernels according to this metric? Out-of-the-box solutions cannot do this. – whuber Oct 31 '13 at 7:31
  • @whuber True, the out of the box systems usually don't have the capability to account for both cost surfaces and kernel density at the same time. My suggestion would be to use the friction surface to calculate the point-to-point distance, then directly apply a kernel function for the weight. In qGIS or ArcGIS you could write use the normal distribution function in python, and in R there's the dnorm() function. – JasonRDalton Sep 21 '14 at 10:08
  • I have done exactly that in some cases, but thinking over the process and inspecting the results reveal some inherent problems. The hardest seems to be how to guarantee that mass is conserved: you cannot just spread values according to say, a fixed Gaussian function, because the total resulting mass of a spread-out point will not equal the original mass. There is no efficient way to perform this computation. – whuber Sep 22 '14 at 13:44

I suggest you try spatstat package for R. There you may set an owin object to determine borders of the study area. Also there is a great tutorial for this package.


I'm actually helping with AniMove plugin for QGIS, that aims to get rid on R dependence for kernel density estimation. Take a look here.

EDIT: The plugin is actually available as experimental in the official QGIS plugin repository

And don't hesitate to ask on the list, if you have any suggestion


I have successfully done this using Geostatistical Analyst which is an extension of ESRI ArcGIS. You can load line features dataset as barriers and the results are pretty decent. I do test runs while changing the parameters of the functions to get a clear idea of how to calibrate the tool. To validate the results, what I recommend is if you have a big enough dataset is remove a sample of points, generate the density surface without those points, then compare the difference between the values of the removed points with the values of the surface at the location of the removed points.

  • 2
    Could you amplify this answer a little to explain how GA--which implements interpolation methods--is able to compute a kernel density estimate? As a double-check of the correctness of your solution, did you verify that the integral of the GA grid was approximately equal to the sum of all the input data? – whuber Feb 11 '13 at 18:20
  • 1
    In 10.1 it is possible to perform kernel interpolation w/ barriers but this does not result in a density estimate. Although, given the methods that the original post references I am wondering if they are not interested in a Gaussian kernel estimate and not a density estimate. – Jeffrey Evans Feb 21 '13 at 22:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.