1

I am trying to load a WFS (address: https://fbinter.stadt-berlin.de/fb/wfs/data/senstadt/s_fb_berlinbtk) into QGIS 3.4 in order to save it as a geopackage. I can connect to the WFS but adding it to my session keeps producing a timeout error message after some of the features have loaded. Other WFS with less features have not caused the same problem. Is there any (possibly quick) way to save a WFS with many features as a geopackage or shapefile?

  • Can you do it in chunks of data? (say bounding box) - that server has many wms/wfs services on it so you might have to be more gentle on the server to get the data. – Mapperz Oct 9 at 14:59
  • As Mapperz says, try dividing the area into several smaller ones. You will get duplicate geometries, but that can be helped after joining the layers. – Erik Oct 9 at 15:06
4

A quick way for doing it without duplicates would be to do the retrieval manually

So

https://fbinter.stadt-berlin.de/fb/wfs/data/senstadt/s_fb_berlinbtk?service=WFS&request=GetFeature&typenames=fis:s_fb_berlinbtk&version=2.0.0&resulttype=hits&

As you say gives us 103331 features:

<wfs:FeatureCollection xmlns:wfs="http://www.opengis.net/wfs/2.0" xmlns:gml="http://www.opengis.net/gml/3.2" xmlns:fis="http://www.berlin.de/broker" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" numberMatched="103331" numberReturned="0" timeStamp="2019-10-09T17:41:10.567" xsi:schemaLocation="http://www.berlin.de/broker https://fbinter.stadt-berlin.de/fb/wfs/data/senstadt/s_fb_berlinbtk?REQUEST=DescribeFeatureType&SERVICE=WFS&VERSION=2.0.0&TYPENAMES=fis:s_fb_berlinbtk http://www.opengis.net/wfs/2.0 http://schemas.opengis.net/wfs/2.0/wfs.xsd http://www.opengis.net/gml/3.2 http://schemas.opengis.net/gml/3.2.1/gml.xsd"/>

Now we can request features incrementally by using the startindex and count paging parameters, like:

https://fbinter.stadt-berlin.de/fb/wfs/data/senstadt/s_fb_berlinbtk?service=WFS&request=GetFeature&typenames=fis:s_fb_berlinbtk&version=2.0.0&startindex=1&count=1000&

for the first 1000 then

https://fbinter.stadt-berlin.de/fb/wfs/data/senstadt/s_fb_berlinbtk?service=WFS&request=GetFeature&typenames=fis:s_fb_berlinbtk&version=2.0.0&startindex=1000&count=2000&

for the next 1000...

You can experiment to see how many requests you can fetch per request, 2000 works but is slow...

You could also do this paging automatically with a little Python.

Then open the XML files in QGIS as vector layers, and join/export as required. Some WFS services give shapefile or sqlite outputs, but unfortunately that isn't available here, the default XML output works though.

First 4000 features from WFS

  • I haven't had the time to look at user30184's answer yet but yours definitely worked. Just a little follow-up question: Why is it that the XML files store data in form of points, lines and polygons? I also managed now to get all of the 103331 features in one go through QGIS. In this case QGIS only displayed polygons. Might it be that the QGIS WFS client simply ignores the points and lines when stored with polygons? – hdi Oct 11 at 12:02
  • On the question of why the WFS is giving data as points, lines and polygons, surely the answer is because that's what the data is, and the GML output allows you to mix those geometry types, whether there's a better way is subjective. As to why QGIS only shows the polygon features, rather than only the points or only the lines, I don't know; but why it only shows one geometry type per layer is because that's how most GIS work. For example shapefiles only allow one geometry type (per shapefile) – nmtoken Oct 11 at 13:21
  • Note if you use the ogr2ogr method ogr2ogr -f gpkg biotypen.gpkg WFS:https://fbinter.stadt-berlin.de/fb/wfs/data/senstadt/s_fb_berlinbtk? fis:s_fb_berlinbtk --config OGR_WFS_PAGE_SIZE 1000 -nln berlinbtk then you get one table (berlinbtk) with mixed geometries, but when you load into QGIS these are portrayed as three separate layers (points, lines, polygons) – nmtoken Oct 11 at 15:05
2

Quick manual way that lets GDAL to take care of the tricks is to use ogr2ogr https://gdal.org/programs/ogr2ogr.html and WFS driver https://gdal.org/drivers/vector/wfs.html.

The basic command for saving the feature type "fis:s_fb_berlinbtk" into GeoPackage "biotypen.gpkg" is

ogr2ogr -f gpkg biotypen.gpkg WFS:https://fbinter.stadt-berlin.de/fb/wfs/data/senstadt/s_fb_berlinbtk? fis:s_fb_berlinbtk

It might be good to increase the default page size of 100 features by adding this parameter to the command:

--config OGR_WFS_PAGE_SIZE 1000

You may want to use some other ogr2ogr parameters as well, like -nln for defining the layer name that is used in GeoPackage.

  • Should have known GDAL could do it, I had in my mind that it didn't support WFS 2 and thus paging, but I see that it does now. One thing I wonder about is the start index. Will GDAL workout whether the service is 2.0.0 and start count at 1, or 2.0.2 and start at 0? – nmtoken Oct 10 at 9:11
  • @nmtoken, from gdal.org/drivers/vector/wfs.html WFS 2.0.2 specification has clarified that the first feature in paging is at index 0. But some server implementations of WFS paging have considered that it was at index 1 (including MapServer <= 6.2). Starting with OGR 1.10, the default base start index is 0, as mandated by the specification. The OGR_WFS_BASE_START_INDEX configuration option can however be set to 1 to be compatible with the server implementations that considered the first feature to be at index 1.. – user30184 Oct 10 at 15:15
  • Perhaps MapServer and other implications got confused by WFS 2.0 standard ( 09-025r1 ) having the default startindex as 1 and not 0, which was as you say was fixed in the corrigendum WFS 2.0.2 as per CR 159 [11-082]. – nmtoken Oct 10 at 16:34
  • The 2.0 was really confusing as you can read starting from github.com/mapserver/mapserver/issues/4180. I would say the default was never meant to be 1. The table in 2.0 did present 1 as the default value but the XML schema had it in another way <xsd:attribute name="startIndex" type="xsd:nonNegativeInteger" default="0"/>. – user30184 Oct 10 at 18:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.