14

Are there any definitive research papers, web resources, or (text)books on distributed GIS? Topics I'm interested in are:

  • Parallelism of sequential spatial algorithms
  • Frameworks for implementing GIS in either the cloud/cluster environment
  • FOSS and commercial software for the above

I have seen this question, but I want something a bit more definitive. CW status would be beneficial, as well as recommendations for tags.

EDIT: I have entered a number of these terms on Google Scholar, and while I am a graduate student and deal with research papers on a regular basis many of the results are not substantive or encompassing.

2
  • You are going to find not a lot of solid information here. The majority of the work in this area is adhoc or highly custom work for governmental clients who need mega-processing. At my agency we have taken ESRI code and made it scale more for our needs but it requires a great deal of care.
    – D.E.Wright
    Commented Feb 22, 2012 at 23:52
  • I implicitly assumed that, as much of my searching has not yielded a solid, vetted resource. I am actively involved in researching some of these items, and so far I have had to meld my academic training with these concepts with little guidance. I am also involved in some consulting with a super-computing center, and they are interested in developing scalable GIS solutions. See my edit above.
    – Baskinomics
    Commented Feb 23, 2012 at 1:34

2 Answers 2

Reset to default
7

I have done several projects in this regard, but at the end they always ended up being custom solutions that basically

  • separated the problem in grids
  • did the processing in each individual node and copied the result to a temp table / data store
  • merged all the solutions to a single result table and optionally handled boundary conditions. Handling boundary conditions was only required for cases where features needed to be stitched or massaged (for example when building a topology in a distributed manner, you want the boundary features to snap).

Funny enough, every single time I solved a problem of this nature, I used a different technology, mostly because that was the fad of the time. At the end, what you want is a message passing library that has a job queue manager, spawns processes on the different nodes, and that has some sort of synchronization mechanism (Semapahore's, Barrier's, etc) and some way to handle errors and retry. All the projects I mention below have these things.

For geo, I have used (in chronological order):

(old school)

(newer)

There are several others that people use.

At the end, any of these would cut it (albeit in a slightly different way), so it boils down to picking one that has a community that is helpful.

Also, on every node, you would have to use a library to do the GIS geometry operations. The recommendation for which one to use would come based on the language that you prefer to code this in. But I am sure the options that most people would give you would be:

Or some other library that serves as wrappers to one of these.

Improve this answer
2
  • 1
    Nice! Currently I am learning Apache Hadoop MapReduce and CGAL, that's really what I had planned on attempting first. I had been holding onto this because I didn't want to flood my question with resources I already in my possession, but I found this useful: nathankerr.com/projects/parallel-gis-processing/…
    – Baskinomics
    Commented Feb 23, 2012 at 1:55
  • MapReduce and CGAL are excellent selections. Can't go wrong with them.
    – Ragi Yaser Burhum
    Commented Feb 24, 2012 at 4:46
1

We are developing a MapReduce based system with SQL interface for large scale spatial data management.

More details can be found @ http://confluence.cci.emory.edu:8090/confluence/display/HadoopGIS/Home http://dl.acm.org/citation.cfm?id=2213603

Improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.