Take the 2-minute tour ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.

I have a set of Sites in the thousands. Not all users see same the same Sites on the map - thus ACL. Data is stored in a non-GIS database (mssql, ora, etc) and right now is retrieved with a standard SQL query and returned to client (OpenLayers) with a JSP.

The current code is already optimized to do client-side clustering (so less of a mess on the client, but all the data is there), bounding-box retrieval (so a sub-set of data but have to go to server for every pan/zoom for new data) and even server-side clustering (to reduce data shipped to client).

What I would like to instead is generate images/tiles for all my data, similar to how Google Maps shows tons of little red dots for all the results - http://bit.ly/d73qrw [google maps search for "coffee"] and display on the client. And when clicked-on do a quick ajax call to a WMS service to get info. That's the idea at least.

But here's the problem - I can't just setup a standard WMS service in front of my data because not all users see same data. Is there a way to generate these tiles on the fly or make WMS ACL-aware?

Thank you,

-Vadim

edit - 9/22/2010 - So I found out how google generates the tiles, or rather the technology behind it. They are using Google Fusion Tables. Store N rows in their tables and then if the data is lat/lon aware, the Fusion Table product can generate the tiles on the fly (!). This is the kind of thing I'm looking for - performant map with tons of data. But of course I need it to be ACL controlled. Does it make sense to write a custom light-weight implementation of the WMS spec or modify an existing product? Though geoserver seems like a lot to "just" modify to support ACL.

edit - 9/27/2010 - Some more info since adding bounty. My data is in Oracle. Ora spatial is not enabled. Right now the data is extracted at the business level and converted to data, sent to client where the client puts the "dots" on the map. ACL is done at the business logic level, not DB or ActivDir or anything like that. Authentication is simple, but Authorization is not and so had to be captured in code. Would like to know how to best create a WMS service to serve up 1000's of "dots" on the map where each user will see a different subset of dots. Is the answer a CQL_FILTER? But then how are the params set? One idea that I have now is to do a 2-step process. First run in-house query to get list of ID's that user allowed to see, then build a WMS request string with those ID's in CQL_FILTER param. Is there anything that simplifies this process? And if I go with this, how can I add this layer as a "WMS" layer to an Open Layers client since to OL the end-point is my code to get IDs from DB not the actual WMS service on e.g. GeoServer?

share|improve this question
    
@list of IDs in CQL_FILTER: If this list gets longer, performance of Geoserver will become really poor. At least thats my experience. Similar question: gis.stackexchange.com/q/1654/187 –  underdark Sep 27 '10 at 19:50
    
Just want to say, we have wrestled with a very similar problem, and I'm encouraged to hear others talking about it. We have some ideas but each has its own tradeoffs. I hope to be coming back to this thread to contribute and/or learn something. –  LarsH Oct 1 '10 at 20:43
add comment

9 Answers 9

Would a Web Feature Service (WFS) be more efficient?

http://openlayers.org/dev/examples/getfeatureinfo-popup.html

Just an idea.

share|improve this answer
2  
still have the issue of 1000's of points, right? while with WMS, 1 image can have 1 site or 1000 sites and performance on client would be the same. –  Vadim Sep 1 '10 at 20:49
add comment

I think that a common solution is to create a proxy or wrapper script that sits between the client and WMS. The proxy is used to craft a custom WMS call based on parameters from the client.

You haven't mentioned whether your purpose for this setup is security or just customization, or what information the client will have to determine which custom 'view' of the data the WMS should produce.

If you will be displaying more than a few hundred points, you will want to stick to an image based service like WMS. Depending on your requirements, you could also just put MapServer behind a wrapper script that makes custom map requests based on unique filter or expression parameters.

share|improve this answer
    
David, The purpose is customization. Security too, but let's assume that's taken care of. But what the user sees is dependent on his/her user rights. So they (1) login, (2) open the map and then see Sites that they are allowed to see. Can you expand more on the idea of MapServer/GeoServer (I'm a Java guy) behind a wrapper that requests with filter. Do you literally mean the "FILTER" or "CQL_FILTER" parameter or something else? I've been researching the CQL_FILTER option since I posted, but haven't been able to test it anywhere online. –  Vadim Sep 2 '10 at 19:52
    
+1. Very sound advice. –  ChristopheD Sep 22 '10 at 19:21
add comment

Likely this solution is crazy, but here we go.

Why not a WMS server per user? Let's suppose we are using MapServer. MapServer configuration is stored in a .map file (aka mapfile), but nothing forbids using several mapfiles. When a WMS request is done, one of the vendor-specific parameters MapServer will accept is 'map', which is a string containing the path and filename of the used mapfile[1]. If in your WMS requests a different mapfile is sent depending on the id of the user who has previously logged in, then you will get as WMS servers as users, and only with one installation of MapServer.

On the security side, you would have to check the used mapfile against a session id, task that would likely be done by a proxy between clients and MapServer. A user wanting to see other user's maps could manually alter the URL changing the 'map' parameter, and this check counter-measure would make such attempts unsuccessful.

The contra (craziness) of this solution is for every user a mapfile must be generated. Obviously, in case the number of users is high this solution is not practical, unless mapfiles could be automatically generated and updated, when needed, by using a script or similar mechanism.

[1] If using MapServer, the 'map' parameter is actually mandatory.

share|improve this answer
    
I appreciate the long write up, however the solution is not practical as the system has many users that also can be added or removed at any time and their data viewing privileges/filters changed at any time by an admin account as well. –  Vadim Sep 22 '10 at 12:42
add comment

There are several moving parts here.

First, there is filtering the points: This should be done on the server side, immediately after or during your SQL query.

Then the filtered points need to be returned to the client.

That is the simple part. :) I would recommend you implement this then test performance, and ideally would be able to stop here.

However, if you have too many points to display efficiently in OpenLayers, then you get to the optimization(s) that google appears to do for the little red dots:

  • You need to connect a tile renderer to the full set of filtered results, producing tiles entirely transparent except for a little dot per result. (Design decision: produce these eagerly, or on demand only?)

    • GeoServer, etc, will require 'trigger' code as they will need a new layer per query per user; as you add users, you'll need to add new things for GeoServer to render. (Also, you'll need to produce result records in a format that can be consumed by the rendering implementation). And how frequently are old tiles re-rendered?

    • I would recommend a small bit of server code that proxies access to the tile renderer -- that way someone won't be able to see results that aren't theirs by guessing a URL.

  • Next, the results that are being returned to the browser need to be partitioned: The first n are full results that get markers placed, while the remainder are sent as coordinates + callback URL. (Even if it is not in charge of rendering them, the browser needs to know where they're located in order to change the cursor, provide a tooltip, and have meaningful action take place upon a click!)

  • Then there's the JS code in the browser to manage interactions w/ the dots.

As far as I know, which is not very far, nobody has written 'out of the box' code to do the above: You're looking at a bunch of systems integration code on the backend and quite a bit of new JS on the frontend.

share|improve this answer
    
Thanks Dan. A lot of "home cooking" is what I expected, but really wanteed to know if there's a better or more elegant solution. Old tiles should be re-rendered frequently as this is part of an EMS and data is constantly updated. I already have client and server-side clustering enabled. And it helps a ton. Because 1000's of points become just 30-40 where most of them are clusters (that I render with a "plus" sign). On a zoom/pan new data is loaded from server through BoundingBox strategy. But clustering removes "clutter" from the map. So you loose the visual of how much stuff is there. –  Vadim Sep 27 '10 at 17:29
    
Forgot to add. In some ways "clutter" is a good thing. And we don't want to represent it with just a variable-sized circle (e.g. big circle for 50 points in cluster, and small for 10) because for our use case it doesn't have the same meaning as have lots of little dots on the map. sigh :) –  Vadim Sep 27 '10 at 17:34
    
Personally, I despise point clustering: it removes a huge amount of information and has no good interpretation at all. So I'm there with you. Also, if your maps are in a simple projection like the typical web map Spherical Mercator: you might consider doing the tile rendering yourself. (It's not much code to convert lat/lon into pixel X/Y coordinates for a tile; I think you can find code on the OpenStreetMap) --- that'll get rid of a lot of the systems integration, redirecting, etc. work. (at the expense of less features, e.g. you can't make the dots as fancy!) –  Dan S. Sep 27 '10 at 17:54
add comment

Another idea (I'm not sure if this is really practical). What you need is a mapping between user permissions and generated map files. So, why not create a hash from user permission and requested map tile/extend? This hash maps to a generated tile. If the map is not there it obviously needs to be generated, otherwise it can be retrieved from the map cache. Of course, this approach does not bring any benefits if each user sees a completely different set of data. But if all users belong to a limited group of permission sets, this should really help.

Daniel

share|improve this answer
add comment

If you have a query that can be executed to tell you which IDs a user can see (as you say in the latest edit) then the "best" solution will probably be to encode that filter as a CQL or OGC filter as part of the WMS request. You may find that reorganizing the data into a view is necessary to make this feasible.

Before you start to use Tile Caches I'd build the simple version and see if it is fast enough, and then go complex if and only if you need more speed.

share|improve this answer
add comment

What is your deadline? In Geomajas, we already have the security / data access limitations. We are working on caching/rasterizing at this moment which will result in the kind of system you are looking for. It should be finished in a couple of weeks.

share|improve this answer
    
hmmm. I'd be interested to hear more. We're in no rush. As I mentioned above (I know, there's a lot to read) we already have a client + server clustering solution working. But I'd like to convert it to a tile-based solution similar to google maps tiles with lots of dots on a tile. How does your product tie into an existing but custom security model? We're not on oAuth or Activ Dir or like that. –  Vadim Oct 1 '10 at 15:35
add comment
up vote 2 down vote accepted

I have an answer that worked for us after some trial and error.

Oracle Spatial + Oracle VPD does the trick. We're already using Ora, and VPD, so this was the next logical step. Ora spatial has a WMS service and can be customized to be different for different users based on their rights/ACL via VPD. If others are looking for something like this, I realize not every operation has or can afford Oracle, but if you have it - it's in there. Don't know if MS SQL Server has similar functionality.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.