Take the 2-minute tour ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.

I'm pretty comfortable with postgis, but I realized that for part of my application, having a database is pretty heavy. I'd like to be able to take a stream of Point objects and check against an in memory collection of Polygon objects for intersection. With postgis, this would be accomplished with ST_Contains().

The biggest reason for moving away from a database is I need real time analysis of the Points as they come in. Right now, I collect a series of Points then do batch work on them, but I find my system struggles to keep up with the input sometimes, mostly because of the I/O associated with accessing the database rather than handling the Points in real time.

I've come across GEOS, but I'm not really sure how to get started using that library.

I'm on a Mac but also have access to a Windows machine if there is a C# solution out there.

EDIT

It's been wisely pointed out that it might be more beneficial to state my problem with my current setup before trying anything too radical.

We collect 1,000 points per second of x,y coordinates. We map these coordinates to zones (polygons) based on which zones a point intersects with (we can have overlapping zones, and thus a point can belong to more than one zone). Once we've established which zone a point is in and which sensor reported that point, we apply our business logic to determine if any actions need to take place.

My first approach was to use the database (Postgres 9.3/postgis 2.1) to check every point as I received it. So I would get a reading from a sensor, figure out which sensor sent me the reading, then figure out which zones that point was in, then apply my logic. I could check a point in about 3 miliseconds and was therefore behind the 8 ball when collecting 1,000 points/second.

My next approach was to batch these zone checkings and business logic interpretations. Now, I grab whatever new data the sensors have produced (usually 5-10 points), figure out which zone each point is in in parallel using the database, then apply my business logic to all those points, again in parallel thanks to the database. Then I poll for new data and repeat.

This approach seems to work well, but is a hog on the system and very fragile. I'm barely keeping up with the data. Someone on the postgresql IRC channel pointed out I'm seeing a lot of overhead writing points to my database just to pull them out miliseconds later to do calculations on those points. They suggested I look at real time steam processing, and that's what I'm trying to investigate now.

I'm thinking if I can read the raw sensor data and clean it before inserting it into the database (identify which zone the point is in and which sensor sent me the reading), I can then apply my business logic in the way I currently do, I should just have more wiggle room.

share|improve this question
    
OGR supports in memory layers, although you may need to implement your own indexing. It may be better to get some detail as to why exactly PostGIS system is failing as it may be a result of your approach rather than a problem with the software itself. –  Matthew Snape Nov 7 '13 at 21:34
2  
Use of an in-memory spatial database (SpatiaLite) could be an option, but without some more info, its hard to provide anything other than opinions. If you're mainly working with points but have high throughput requirements, you might be better off using an existing RTree* implementation and wiring that into your business needs. –  BradHards Nov 7 '13 at 21:55
    
@MatthewSnape - I've updated the question with some more background info. Let me know if that helps. Thanks for the suggestion. –  Tyler DeWitt Nov 7 '13 at 22:45

2 Answers 2

up vote 8 down vote accepted

If you're comfortable with C/C++, GEOS: http://trac.osgeo.org/geos

If you're comfortable with C#, NTS: http://code.google.com/p/nettopologysuite/

If you're comfortable with Java, JTS: http://tsusiatsoftware.net/jts/main.html

If you're comfortable with Python, shapely: https://github.com/Toblerity/Shapely

If you're comfortable with Ruby, ffi-geos: https://github.com/dark-panda/ffi-geos

If you're comfortable with Javascript, JSTS: http://github.com/bjornharrtell/jsts

If you're not comfortable, buy a better chair.

share|improve this answer
1  
+1 for the chair recommendation –  Tyler DeWitt Nov 8 '13 at 14:31
    
+1 If you are using Javascript, JSTS(port of JTS) github.com/bjornharrtell/jsts –  agate Mar 13 at 2:49

I'd suggest picking up on using Python with the Rtree and shapely modules.

Rtree uses an R-tree index, which can be used to very quickly a point to be contained in a bounding-box of what you describe as a zone, then shapely can check to see if the point is actually within the polygon of each zone (rather than just within the bbox). Shapely uses GEOS library just like PostGIS.

The advantage of writing this in Python over C# is that it is portable between Mac and Windows.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.