Use custom projection in MapProxy

Question

I'm trying to use the proj_data_dir config setting in MapProxy to add support for a custom projection. So far I've been unsuccessful and suspect I'm almost there, but not quite...

I've created a file named "epsg" in a directory also called "epsg". The content of the epsg text file is:

# Custom projection
<100000> +proj=lcc +lat_1=-25 +lat_2=-32 +lat_0=-28.5 +lon_0=24.5 +x_0=0 +y_0=0 +datum=WGS84 +units=m +no_defs <>

# Standard EPSG's we're also using in the *.yaml config file
# Copied from http://epsg.io/
<3857> +proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext  +no_defs <>
<27200> +proj=nzmg +lat_0=-41 +lon_0=173 +x_0=2510000 +y_0=6023150 +ellps=intl +towgs84=59.47,-5.04,187.44,0.47,-0.1,1.024,-4.5993 +units=m +no_defs <>

My mapproxy.yaml file has the following contents:

services:
  demo:
  tms:
    use_grid_names: true
  kml:
      use_grid_names: true
  wmts:
  wms:
    srs: ['EPSG:3857','EPSG:100000']
    md:
      title: MapProxy WMS Proxy
      abstract: This is a minimal MapProxy example.

layers:
  - name: satellite
    title: Mapbox Satellite
    sources: [satellite_cache]
  - name: satellite_custom
    title: Mapbox Satellite reprojected as custom map grid
    sources: [satellite_custom_cache]
    
caches:
  satellite_cache:
    grids: [mapbox_grid]
    sources: [satellite_tiles]
  satellite_custom_cache:
    grids: [custom_grid]
    sources: [satellite_cache]
    
sources:
  satellite_tiles:
    type: tile
    url: https://api.mapbox.com/v4/mapbox.satellite/%(z)s/%(x)s/%(y)s.png?access_token=[REDACTED]
    grid: mapbox_grid
    
grids:
  webmercator:
    base: GLOBAL_WEBMERCATOR
  mapbox_grid:
    base: GLOBAL_MERCATOR
    srs: 'EPSG:3857'
    origin: 'nw'
  custom_grid:
    srs: 'EPSG:100000'
    origin: 'nw'
    
globals:
  srs:
    proj_data_dir: 'epsg\'

And then when I attempt to run MapProxy, I get an error that suggests it uses the epsg directory correctly, but maybe isn't reading the epsg text file it contains:

pyproj.exceptions.CRSError: Invalid projection: epsg:100000: (Internal Proj Error: proj_create: crs not found)

The full error report:

C:\Python>mapproxy-util serve-develop mapproxy.yaml
[2021-02-12 11:44:45,052] mapproxy.config - INFO - reading: C:\Python\mappoxy.yaml
[2021-02-12 11:44:45,058] mapproxy.system - INFO - loading proj data from C:\Python\epsg\
Traceback (most recent call last):
  File "c:\python\python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\python\python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Python\Python39\Scripts\mapproxy-util.exe\__main__.py", line 7, in <module>
  File "c:\python\python39\lib\site-packages\mapproxy\script\util.py", line 387, in main
    commands[command]['func'](args)
  File "c:\python\python39\lib\site-packages\mapproxy\script\util.py", line 85, in serve_develop_command
    app = make_wsgi_app(mapproxy_conf, debug=options.debug)
  File "c:\python\python39\lib\site-packages\mapproxy\wsgiapp.py", line 58, in make_wsgi_app
    services = conf.configured_services()
  File "c:\python\python39\lib\site-packages\mapproxy\config\loader.py", line 211, in configured_services
    return self.services.services()
  File "c:\python\python39\lib\site-packages\mapproxy\config\loader.py", line 1855, in services
    new_services = creator(service_conf or {})
  File "c:\python\python39\lib\site-packages\mapproxy\config\loader.py", line 2018, in demo_service
    lyr = layer_conf.wms_layer()
  File "c:\python\python39\lib\site-packages\mapproxy\util\py.py", line 78, in wrapper
    cache[key] = func(self, *args, **kwargs)
  File "c:\python\python39\lib\site-packages\mapproxy\config\loader.py", line 1683, in wms_layer
    map_layer = self.context.caches[source_name].map_layer()
  File "c:\python\python39\lib\site-packages\mapproxy\util\py.py", line 78, in wrapper
    cache[key] = func(self, *args, **kwargs)
  File "c:\python\python39\lib\site-packages\mapproxy\config\loader.py", line 1599, in map_layer
    for grid, extent, tile_manager in self.caches():
  File "c:\python\python39\lib\site-packages\mapproxy\util\py.py", line 78, in wrapper
    cache[key] = func(self, *args, **kwargs)
  File "c:\python\python39\lib\site-packages\mapproxy\config\loader.py", line 1503, in caches
    sources, source_image_opts = self._sources_for_grid(
  File "c:\python\python39\lib\site-packages\mapproxy\config\loader.py", line 1376, in _sources_for_grid
    tile_grid=grid_conf.tile_grid(),
  File "c:\python\python39\lib\site-packages\mapproxy\util\py.py", line 78, in wrapper
    cache[key] = func(self, *args, **kwargs)
  File "c:\python\python39\lib\site-packages\mapproxy\config\loader.py", line 300, in tile_grid
    grid = tile_grid(
  File "c:\python\python39\lib\site-packages\mapproxy\grid.py", line 101, in tile_grid
    srs = SRS(srs)
  File "c:\python\python39\lib\site-packages\mapproxy\srs.py", line 96, in SRS
    srs = _srs_impl(srs_code)
  File "c:\python\python39\lib\site-packages\mapproxy\srs.py", line 320, in __init__
    self.proj = CRS.from_epsg(epsg_num)
  File "c:\python\python39\lib\site-packages\pyproj\crs\crs.py", line 331, in from_epsg
    return CRS(_prepare_from_epsg(code))
  File "c:\python\python39\lib\site-packages\pyproj\crs\crs.py", line 296, in __init__
    super().__init__(projstring)
  File "pyproj/_crs.pyx", line 2302, in pyproj._crs._CRS.__init__
pyproj.exceptions.CRSError: Invalid projection: epsg:100000: (Internal Proj Error: proj_create: crs not found)

Any ideas what I'm doing wrong? I imagine the epsg file isn't being read. Is there a different format I should be using? Unfortunately there seems to be very little in the way of concrete examples about how to use the proj_data_dir setting and exactly what it expects to find in the directory.

The closest thing to a concrete example I can find is from this old discussion: https://lists.osgeo.org/pipermail/mapproxy/2013-July/001628.html

And the MapProxy Configuration document states:

proj_data_dir

MapProxy uses PROJ for all coordinate transformations. If you need custom projections or need to tweak existing definitions. You can point MapProxy to your own set of PROJ data files.

This path should contain an epsg file with the EPSG definitions for installations with PROJ version 4. PROJ>=5 uses a different configuration format. Please refer to the PROJ documentation.

The configured path can be absolute or relative to the mapproxy.yaml.

Further investigations:

Further investigations seems to suggest that the proj_data_dir option does work but will only consume projections from the proj.db file. All other files alongside proj.db seem to be ignored, but this seems to be because MapProxy won't allow srs authorities other that EPSG.

It looks like MapProxy originally specified using the "epsg" filename for custom projections as the filename must be used as the srs authority by pyproj. However it seems that pyproj won't serve up EPSG codes that come from anywhere but its proj.db file now.

As a work-around I was hoping to use a non-EPSG authority to serve up my custom projection, dropping in an "xxxx" file so I could use an srs of 'XXXX:10000'. Unfortunately MapProxy seems to take any srs provided in the mapproxy.yaml file and change it to 'EPSG'. I tried to use 'XXXX:10000' but MapProxy automatically changed it to 'EPSG:10000', going by the error message.

Seems it will also do this with existing non-EPSG authority projections, such as ESRI, that are held within proj.db and try to use an EPSG equivalent instead. For example MapProxy can't use the existing 'ESRI:102024' projection as it tries to request 'EPSG:102024' instead and so gets the following error pyproj.exceptions.CRSError: Invalid projection: epsg:102024: (Internal Proj Error: proj_create: crs not found).

So the only work-around I can think of is a horrible hack to try and work out how to convert my projection string into the equivalent table entries in proj.db. Unless anyone has an alternative approach I could use?

Answer 1

The only solution I found so far is to edit proj.db directly to add the custom projections.

It looks like the pyproj library has evolved over time but MapProxy hasn't caught up with its changed file conventions yet.

I've submitted a bug report to MapProxy - https://github.com/mapproxy/mapproxy/issues/493

The easiest way I found to add a new projection to proj.db was to open it up in some SQLite tool and add a new entry to the projected_crs table providing a WKT definition in the text_definition column.

Example SQL insert statement:

INSERT INTO projected_crs (
  auth_name,
  code,
  name,
  description,
  coordinate_system_auth_name,
  coordinate_system_code,
  geodetic_crs_auth_name,
  geodetic_crs_code,
  conversion_auth_name,
  conversion_code,
  text_definition,
  deprecated
)
VALUES (
  'EPSG',
  '100000',
  'CUSTOM ENTRY 1',
  NULL,
  NULL,
  NULL,
  'EPSG',
  '4326',
  NULL,
  NULL,
  'PROJCS["CUSTOM ENTRY 1",GEOGCS["GCS_WGS_1984",DATUM["WGS_1984",SPHEROID["WGS_1984",6378137,298.257223563]],PRIMEM["Greenwich",0],UNIT["Degree",0.017453292519943295]],PROJECTION["Lambert_Conformal_Conic_2SP"],PARAMETER["False_Easting",0],PARAMETER["False_Northing",0],PARAMETER["Central_Meridian",25],PARAMETER["Standard_Parallel_1",-25],PARAMETER["Standard_Parallel_2",-32],PARAMETER["Latitude_Of_Origin",-28.5],PARAMETER["Longitude_Of_Origin",24.5],UNIT["Meter",1],AUTHORITY["EPSG","100000"]]',
  '0'
)