8

I've inherited a pretty large / complex (internal) web portal (run under python 2.6.6), which has some mapping capabilities, using Mapnik 2.2.0. The web service has recently started throwing 500 errors when it tries to create some of the maps that are displayed, with the sole error:

$ Premature end of script headers: app.wsgi

The maps are generated from geojson that is passed as a string (having been created from a python dictionary); I've tracked it down to 1 line of code in a map.py, which adds a layer to a map:

data = mapnik.Ogr(file=json, layer='OGRGeoJSON')

when I comment out that line of code (and the associated m.layers[-1].data = data) then the image is displayed on the portal, but obviously without this layer. The json is valid (I have checked it on http://geojsonlint.com).


Debugging efforts

Using a separate json file

I wanted to see whether calling the JSON from an external source replicated the problem. I therefore saved the json string as a separate file (test.json), in the same directory as map.py, and modified the Ogr call to

data = mapnik.Ogr(file='test.json', layer='OGRGeoJSON')

I still get a 500 error, but with a slightly more verbose error message:

datasource = mapnik.Ogr(file='testgeo.json', layer='OGRGeoJSON')
File /usr/lib64/python2.6/site-packages/mapnik/__init__.py, line 536, in Ogr
return CreateDatasource(keywords)
RuntimeError: OGR Plugin: Failed to read GeoJSON data

Checking the geojson

$ ogrinfo testgeo.json
ERROR 4: GeoJSON Driver doesn't support update.
Had to open data source read-only.
INFO: Open of `testgeo.json'
      using driver `GeoJSON' successful.
1: OGRGeoJSON (3D Polygon)

So the json seems valid


Using the python command line

I have also tried running python interactively in the same directory as map.py and test.json:

$ python
Python 2.6.6 (r266:84292, Jul 23 2015, 15:22:56)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-11)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import mapnik
>>> mapnik.Ogr(file="testgeo.json", layer="OGRGeoJSON")
<mapnik.Datasource object at 0x7f6243c27398>

Further info

$ gdal-config --version
1.9.2

$ ogr2ogr --version
GDAL 1.9.2, released 2012/10/08

This returns with no errors (so I assume that this means the geojson is valid, confirming my earlier test of running the geojson through http://geojsonlint.com):

$ ogr2ogr testgeo.shp testgeo.json

OS Version

$ lsb_release -a
LSB Version:    :base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch
Distributor ID: CentOS
Description:    CentOS release 6.7 (Final)
Release:        6.7
Codename:       Final

These 3 debugging efforts show that the json is valid and that the call works, except when it's run through the webstack. I've checked the permissions and ownership of all relevant files and directories, and they all seem to be as I'd expect.

What else could be causing the issue?!


Update (2 weeks after asking the original question)

I have come into work today and thought I'd have another crack at looking to see if I could tease some error messages out (new week, fresh start). But, utterly bizarrely, the code does not seem to be failing (for now, at least). I have changed nothing, and as far as I can gather, there have been no hardware changes to the servers.

This now becomes an investigation of what did cause it, and whether there's anything I can do to prevent it happening again...

  • I'm not a developer but at a first sight I see some difference in the OGR synthax definition of the json data (e.g. ' ' ' instead of ' " '). Furthermore I remember that the OGR supported file extension is ".geojson" instead of ".json". Hope it can help you in some way – MickyInTheSky Feb 2 '16 at 11:58
  • Unfortunately, when I use an external file it does work (regardless of the file extension); it's when the json is embedded in the python as a string it has now stopped working – ChrisW Feb 2 '16 at 13:36
  • Have you checked, you use the same version of python/ mapnik scripts to execute mapnik.Ogr in the commandline and your webstack? Maybe there's a different implementation and an update could help? – tareq Feb 2 '16 at 13:46
  • Yes, it's Mapnik 2.2.0 and Python 2.6.6 in all cases – ChrisW Feb 2 '16 at 13:47
  • 1
    What is the OS that this is being deployed to? Once I had a weird bug going on, because of permissions. this was the rough structure: folder A > folder B (where B is child of A). Then, I had given all permissions to folder B, but not A. Since A was not accessible to the process, everything else failed (I had used -R as an option). If I'm not mistaken, this was on RHEL. Double check that, perhaps, it's a thing. – George Silva Feb 3 '16 at 15:15
2
+50

wait, it looks like it's expecting a file or a file-like object. when you say you have an json embedded in python, it's the whole json text, not the file path?

a = "/home/george/boundaries.json" # file-path
b = "{
    'type': 'FeatureCollection',
    'crs': {
      'type': 'name',
      'properties': {
        'name': 'EPSG:3857'
      }
    },
    'features': [{
      'type': 'Feature',
      'geometry': {
        'type': 'Point',
        'coordinates': [0, 0]
      }
    }, {
      'type': 'Feature',
      'geometry': {
        'type': 'LineString',
        'coordinates': [[4e6, -2e6], [8e6, 2e6]]
      }
    }]}" # json-content

Mapnik API it's expecting a file and it will open it up and parse it by itself.

Check the reference: http://mapnik.org/docs/v2.1.0/api/python/mapnik-module.html#Ogr

Copied from there, for completion:

from mapnik import Ogr, Layer
datasource = Ogr(base='/home/mapnik/data',file='rivers.geojson',layer='OGRGeoJSON') 
lyr = Layer('OGR Layer from GeoJSON file')
lyr.datasource = datasource

This link is for older docs (2.1) but my guess is that the API did not changed.

Of course, this is assuming, by your comments, that you're using option B, instead of A.

  • Sorry, my question obviously isn't clear enough. The original application is using a json string embedded within python - this stopped working. As part of my debugging efforts I extracted the json to make it use an external file. This only works when you call it on the python command line, but not through the webstack – ChrisW Feb 3 '16 at 12:08
  • oh ok. on a comment you said that it would always work on external files, regardless of extension. that's why I thought you were using the option B I showed in the answer. Can you do an ogr2ogr to see if everything is ok with your json? What versions are your underlying libraries? – George Silva Feb 3 '16 at 12:17
  • I meant that when I was testing with external files I didn't find any difference between whether the file extension was .json or .geojson; I've never used ogr2ogr - but it seems like it needs an input file, which I don't have. Which libraries should I be checking? – ChrisW Feb 3 '16 at 12:19
  • GDAL/OGR. try to convert your json file to let's say, a shapefile, using ogr2ogr. if it all goes all right, it's a indication that the problem is not in GDAL/OGR, which is used by mapnik. the permissions of the file are all ok? this comes down to debugging now. I once used mapnik in a webserver to generate a map, and never had this problem. – George Silva Feb 3 '16 at 12:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.