We are implementing WFS using geoserver for our customers to consume vector data from a hosted spatial database to their desktop GIS. They want to be able to just connect and browse the data in the same was that they would to for example an SDE connection. Some of the datasets they want to consume are very large, and contain way more features than the feature limits we want to set on the server. They may therefore not get all features downloaded, and if features are missing the user isn’t aware that they have hit the feature limit and don’t have all the features.
If we just increase feature limits, performance degrades, and the client still may not get all features, unless the limit is set larger than feature count in the data.
Basically we are experiencing performance and usability issues around guaranteeing that all features are downloaded for large datasets to desktop clients.
We are testing with Quantum, Arc9 and 10 with Data Interoperability connection.
So far we have been investigating how the caching options on the client can help. We are coming to the conclusion that caching can help, but this still doesn't guarantee all features are always downloaded.
In quantum turning caching off and using scale thresholds helps, as features are requested again when the client zooms/pans if, but only if, the new envelope is outside the initial envelope.
In Arc 9 Data Interopeability, there is no ability to turn caching off, and no ability to restrict to an envelope, so the client gets the features on the first request for the whole dataset, limited by geoserver feature limit and never re-requests during the life of the cache(24hrs), so if features are missing, they will stay missing.
In Arc10 Data Interopeability, it's possible to use a live connection, which re-requests features on every pan/zoom, but as we increase feature limit on server, performance degrades. Also, for certain tasks (view attribute table, select), arc10 requests all features one by one, which is pretty much un-usable.
In all cases, there are scenarios where the feature limit has been hit, features are missing, but the user isn’t aware because the client doesn’t flag this.
I would like some idea how people have approached serving large datasets over WFS to desktops in a way that guarantees the client gets all the features for their view extent. Do we need to improve performance on the server and up the feature limit? What are people doing to tackle this?
I would also like to try and get to a better understanding of what WFS is for. Is WFS even appropriate for looking at large datasets in a desktop session as users are used to doing with shapefiles or a geo-database connections?
I realise this question is wide open, so any views much appreciated.