I'm trying to modify create_utfgrids.py to use an OGR VRT file, rather than shapefiles, and while the OGR stuff in this code responds to the VRT, the Mapnik stuff is not. It's failing with..

ERROR 1: Failed to open datasource `PG:host= user=postgres dbname=gis password=passw0rd tables=parcels_20140829_pmerc'.

I believe the VRT is fine, because it not only works with ogrinfo directly, but as stated above, the OGR-related code in the Python file also works.

While I'm pretty confident the VRT isn't the issue, for those who might ask, the VRT I tried looks like this. I know there's some redundant stuff in there, but that's because I thought maybe Mapnik was looking for something specific, so I was adding any node I could think of which might be relevant:

  <OGRVRTLayer name="parcels_20140829_pmerc">
    <SrcDataSource>PG:host= user=postgres dbname=gis password=passw0rd tables=parcels_20140829_pmerc</SrcDataSource>
    <SrcSQL>SELECT ST_AsBinary(wkb_geometry) as geomm, tms, owner_name, the_wkt FROM parcels_20140829_pmerc</SrcSQL>
    <GeometryField encoding="WKB" field="geomm"></GeometryField>
    <Field name="tms" src="tms" type="String"/>
    <Field name="owner_name" src="owner_name" type="String"/>
    <Field name="the_wkt" src="the_wkt" type="String"/>

I eventually got past the problem by passing in an OGR PostgreSQL connection string, parsing over that, and then pushing the values into a mapnik.PostGIS() constructor. Sure, this solves my immediate problem, but it's not quite as generic/universal as I would like.

So I'm wondering, since I can't get Mapnik to like my VRT, if it's possible to reference the existing ogr.Open() datasource object from Mapnik and avoid having to use something like mapnik.PostGIS() or mapnik.Ogr() in the first place.

To get this idea, this is kind of how I imagine it..

import mapnik
import ogr

# create_utfgrids.py uses shapefiles, hence "shppath".
# But for this exercise it represents a path to a VRT.
ds = ogr.Open(shppath)

# A bunch of OGR stuff here.

# This is where I want the magic to happen, 
# pushing the OGR dataset into Mapnik..
ds = mapnik.Ogr(ds)

# And then Mapnik stuff follows.

Stepping back, if you have data in PostGIS and you want to render it with Mapnik you really should be using Mapnik's native PostGIS support - aka mapnik.PostGIS datasource in python (https://github.com/mapnik/mapnik/wiki/PostGIS). That will be faster and more efficient than going through OGR. That said, if you have a good reason for connecting to PostGIS through an OGR VRT file then that should work just fine via the Mapnik mapnik.Ogr() datasource. I would recommend getting that working (please file an issue at https://github.com/mapnik/mapnik-support with details on how to replicate) instead of using the python plugin because the python plugin is going to be the slowest of all options.

  • Thanks, Dane. I did get it working using a mapnik.PostGIS() datasource, but for some unknown reason I couldn't get Mapnik to accept the VRT above, which declared a PostgreSQL datasource. When tinkering with it later, I got curious if I could pass in an existing OGR instance.
    – elrobis
    Sep 29 '14 at 14:36

Depending on your version of mapnik, yes it's possible. Mapnik 2.1 introduced the Python Plugin as a data source.

You would need to subclass mapnik.PythonDatasource and implement a features method that based on an incoming mapnik.Query object will return mapnik.PythonDatasource.wkb_features objects.

For example:

import mapnik
import ogr

class VRTDataSource(mapnik.PythonDatasource):

    def __init__(self, shppath, layer_idx=0):
        self.ds = ogr.Open(shppath)
        self.layer_idx = layer_idx
        super(VRTDatasource, self).__init__()

    def features(self, query):
        bounding_box = query.bbox

        layer = self.ds[self.layer_idx]


        return mapnik.PythonDatasource.wkb_features(
            keys = layer[0].keys(),  # Assuming that all features have the
                                     # same table definition
            features = [
                ( feature.GetGeometryRef().ExportToWkb(), dict(feature) ), 
                for feature in layer # OGR layers are feature iterators

# Then when building your mapnik map object

ds = mapnik.Python(factory='VRTDataSource', shppath=path, layer_idx=0)
layer = mapnik.Layer('python')
layer.datasource = ds

The only point I'd worry about would be the Projection - I'm not sure if mapnik will pass the CRS as part of the mapnik.Query object and expect a projected WKB in return, or whether setting the projection of the mapnik layer means mapnik will project on the fly. My guess would be the former, but I'd want to test first.

UPDATE Expand the example code so that you don't need to specify anything beyond the data source and the layer name (assuming that the layer schema is consistent).

  • to the rescue, again! It's always you, man! :) This problem is on my work system, which is Windows, so I'll have to confirm the Mapnik version I'm running tomorrow, as well as the rest of the recipe. Definitely worthy of my +1 in the meantime.
    – elrobis
    Sep 26 '14 at 2:29
  • 1
    Ok @om_henners, I do have Mapnik 2.2.0, but now I'm looking closer at this and I'm not clear how it's really pushing an existing OGR datasource into Mapnik. I see where ogr.Open() happens in the subclass, which does get the datasource, but if I have to hard-code all the attribute fields into the subclass ..I think that defeats the idea. It seems to me the OGR datasource should have everything Mapnik needs, so I anticipated there might be an elegant, decoupled way of saying "Ok Mapnik, take this existing OGR datasource and run." Know what I mean?
    – elrobis
    Sep 26 '14 at 14:30
  • Very Interesting... Sep 26 '14 at 14:51
  • @elrobis Happy to help! I've modified the answer above to pull back fields (and keys) directly from the layer. That said I agree with Dane that long term it would be better to get the VRT layer working with the OGR plugin.
    – om_henners
    Sep 28 '14 at 11:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.