1

Background

I'm working on an application that has multiple threads getting features from the same OGR layer at the same time, but with different spatial filters. This is problematic because it causes collisions. To solve this I have the worker threads ask a different thread to get the features for them and place them in a queue for the workers. This introduces a different problem.

Since I have no control over how many features that might exist on a layer I have to put a limit on how many features to get at a time so as to not let the memory usage run away uncontrollably (I've currently set this limit to 50 per worker). That means that for every worker the feature getting thread has to know how many features it has received and start from there. In other words, it has to execute this code before it starts getting the next 50 features from the layer:

layer.SetAttributeFilter(aFilter);
layer.SetSpatialFilterRect(anEnvelope.MinX, anEnvelope.MinY, anEnvelope.MaxX, anEnvelope.MaxY);
layer.SetNextByIndex(index);

That layer.SetNextByIndex is a massive time sink. OGR does not have an optimized version of that function implemented in its WFS layer class so what it does is reset the reading to zero and do GetFeature() index number of times. layer.TestCapability(Ogr.OLCFastSetNextByIndex); always returns false, as can be seen in ogrwfslayer.cpp.

Question

What I'm wondering is whether OGR doesn't have that function implemented due to a hiccup or laziness or if WFS simply doesn't have that capability.

I'm using GeoServer for the WFS. Is it possible for me to implement the OLCFastSetNextByIndex functionality on my own?

Sources:
http://www.gdal.org/classOGRLayer.html#ac0b0869ff6fcadda35cbac1f3edf8eae
https://github.com/OSGeo/gdal/blob/master/gdal/ogr/ogrsf_frmts/wfs/ogrwfslayer.cpp
https://github.com/OSGeo/gdal/blob/master/gdal/ogr/ogrsf_frmts/generic/ogrlayer.cpp

  • WFS offers some query parameters which may help. Resulttype=hits returns the total number of features after the filter is applied, and with count= and startindex= user can select which features to read. Perhaps GDAL is utilizing those as well. – user30184 May 9 '18 at 12:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.