Checking for Changes in a WFS

Question

I'm responsible for updating many spatial datasets within my organisation. Some data suppliers notify me of a change, some suppliers don't. Sometimes it's not easily apparent if the data has changed at all.

I'm starting to become proficient in pyqgis. It occurs to me I could automate the downloading of a dataset, save it off as a shape, get an area size, number of features (or whatever makes sense) and compare these values to those previously recorded.

This could be scheduled to run at night time to reduce peak load on my organisations and our suppliers hardware.

Before I start, can anyone tell me if there is a more efficient way to do this. Take for example this WFS URL (super light weight at 23 features). I can't see how there could be a better way, but then again I don't know what I don't know.

http://www.geostore.com/OGC/OGCInterface?SERVICE=WFS&UID=UDATAGOV2011&PASSWORD=datagov2011&INTERFACE=ENVIRONMENTWFS&VERSION=2.0.0&LC=200000000000000000

Answer 1

Short answer: I don't believe this is possible using WFS (or any of the OGC standards).

The WFS standard offers no way of knowing how many features are in the entire dataset. Nor does it offer any way of showing data history.

The only real way of knowing if anything is different is by downloading the entire dataset and doing comparisons (spatial and attribute). But again, the WFS standard isn't too well suited to that - most WFS servers will only serve up to a maximum of x features in response to a single request (GeoServer's default is 100,000 I believe).

The only other way I can think of is for datasets that have a column of last_updated or similar, you could potentially do a WFS query including a filter and requesting only features where last_updated is greater than the date you last ran the update.

Finally, this question may be of some interest to see how QGIS deals with WFS - I assume PyQGIS will be similar - Editing WFS with max feature limit in QGis