0

I'm trying to use the proj_data_dir config setting in MapProxy to add support for a custom projection. So far I've been unsuccessful and suspect I'm almost there, but not quite...

I've created a file named "epsg" in a directory also called "epsg". The content of the epsg text file is:

# Custom projection
<100000> +proj=lcc +lat_1=-25 +lat_2=-32 +lat_0=-28.5 +lon_0=24.5 +x_0=0 +y_0=0 +datum=WGS84 +units=m +no_defs <>

# Standard EPSG's we're also using in the *.yaml config file
# Copied from http://epsg.io/
<3857> +proj=merc +a=6378137 +b=6378137 +lat_ts=0.0 +lon_0=0.0 +x_0=0.0 +y_0=0 +k=1.0 +units=m +nadgrids=@null +wktext  +no_defs <>
<27200> +proj=nzmg +lat_0=-41 +lon_0=173 +x_0=2510000 +y_0=6023150 +ellps=intl +towgs84=59.47,-5.04,187.44,0.47,-0.1,1.024,-4.5993 +units=m +no_defs <>

My mapproxy.yaml file has the following contents:

services:
  demo:
  tms:
    use_grid_names: true
  kml:
      use_grid_names: true
  wmts:
  wms:
    srs: ['EPSG:3857','EPSG:100000']
    md:
      title: MapProxy WMS Proxy
      abstract: This is a minimal MapProxy example.

layers:
  - name: satellite
    title: Mapbox Satellite
    sources: [satellite_cache]
  - name: satellite_custom
    title: Mapbox Satellite reprojected as custom map grid
    sources: [satellite_custom_cache]
    
caches:
  satellite_cache:
    grids: [mapbox_grid]
    sources: [satellite_tiles]
  satellite_custom_cache:
    grids: [custom_grid]
    sources: [satellite_cache]
    
sources:
  satellite_tiles:
    type: tile
    url: https://api.mapbox.com/v4/mapbox.satellite/%(z)s/%(x)s/%(y)s.png?access_token=[REDACTED]
    grid: mapbox_grid
    
grids:
  webmercator:
    base: GLOBAL_WEBMERCATOR
  mapbox_grid:
    base: GLOBAL_MERCATOR
    srs: 'EPSG:3857'
    origin: 'nw'
  custom_grid:
    srs: 'EPSG:100000'
    origin: 'nw'
    
globals:
  srs:
    proj_data_dir: 'epsg\'

And then when I attempt to run MapProxy, I get an error that suggests it uses the epsg directory correctly, but maybe isn't reading the epsg text file it contains:

pyproj.exceptions.CRSError: Invalid projection: epsg:100000: (Internal Proj Error: proj_create: crs not found)

The full error report:

C:\Python>mapproxy-util serve-develop mapproxy.yaml
[2021-02-12 11:44:45,052] mapproxy.config - INFO - reading: C:\Python\mappoxy.yaml
[2021-02-12 11:44:45,058] mapproxy.system - INFO - loading proj data from C:\Python\epsg\
Traceback (most recent call last):
  File "c:\python\python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\python\python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Python\Python39\Scripts\mapproxy-util.exe\__main__.py", line 7, in <module>
  File "c:\python\python39\lib\site-packages\mapproxy\script\util.py", line 387, in main
    commands[command]['func'](args)
  File "c:\python\python39\lib\site-packages\mapproxy\script\util.py", line 85, in serve_develop_command
    app = make_wsgi_app(mapproxy_conf, debug=options.debug)
  File "c:\python\python39\lib\site-packages\mapproxy\wsgiapp.py", line 58, in make_wsgi_app
    services = conf.configured_services()
  File "c:\python\python39\lib\site-packages\mapproxy\config\loader.py", line 211, in configured_services
    return self.services.services()
  File "c:\python\python39\lib\site-packages\mapproxy\config\loader.py", line 1855, in services
    new_services = creator(service_conf or {})
  File "c:\python\python39\lib\site-packages\mapproxy\config\loader.py", line 2018, in demo_service
    lyr = layer_conf.wms_layer()
  File "c:\python\python39\lib\site-packages\mapproxy\util\py.py", line 78, in wrapper
    cache[key] = func(self, *args, **kwargs)
  File "c:\python\python39\lib\site-packages\mapproxy\config\loader.py", line 1683, in wms_layer
    map_layer = self.context.caches[source_name].map_layer()
  File "c:\python\python39\lib\site-packages\mapproxy\util\py.py", line 78, in wrapper
    cache[key] = func(self, *args, **kwargs)
  File "c:\python\python39\lib\site-packages\mapproxy\config\loader.py", line 1599, in map_layer
    for grid, extent, tile_manager in self.caches():
  File "c:\python\python39\lib\site-packages\mapproxy\util\py.py", line 78, in wrapper
    cache[key] = func(self, *args, **kwargs)
  File "c:\python\python39\lib\site-packages\mapproxy\config\loader.py", line 1503, in caches
    sources, source_image_opts = self._sources_for_grid(
  File "c:\python\python39\lib\site-packages\mapproxy\config\loader.py", line 1376, in _sources_for_grid
    tile_grid=grid_conf.tile_grid(),
  File "c:\python\python39\lib\site-packages\mapproxy\util\py.py", line 78, in wrapper
    cache[key] = func(self, *args, **kwargs)
  File "c:\python\python39\lib\site-packages\mapproxy\config\loader.py", line 300, in tile_grid
    grid = tile_grid(
  File "c:\python\python39\lib\site-packages\mapproxy\grid.py", line 101, in tile_grid
    srs = SRS(srs)
  File "c:\python\python39\lib\site-packages\mapproxy\srs.py", line 96, in SRS
    srs = _srs_impl(srs_code)
  File "c:\python\python39\lib\site-packages\mapproxy\srs.py", line 320, in __init__
    self.proj = CRS.from_epsg(epsg_num)
  File "c:\python\python39\lib\site-packages\pyproj\crs\crs.py", line 331, in from_epsg
    return CRS(_prepare_from_epsg(code))
  File "c:\python\python39\lib\site-packages\pyproj\crs\crs.py", line 296, in __init__
    super().__init__(projstring)
  File "pyproj/_crs.pyx", line 2302, in pyproj._crs._CRS.__init__
pyproj.exceptions.CRSError: Invalid projection: epsg:100000: (Internal Proj Error: proj_create: crs not found)

Any ideas what I'm doing wrong? I imagine the epsg file isn't being read. Is there a different format I should be using? Unfortunately there seems to be very little in the way of concrete examples about how to use the proj_data_dir setting and exactly what it expects to find in the directory.

The closest thing to a concrete example I can find is from this old discussion: https://lists.osgeo.org/pipermail/mapproxy/2013-July/001628.html

And the MapProxy Configuration document states:

proj_data_dir

MapProxy uses PROJ for all coordinate transformations. If you need custom projections or need to tweak existing definitions. You can point MapProxy to your own set of PROJ data files.

This path should contain an epsg file with the EPSG definitions for installations with PROJ version 4. PROJ>=5 uses a different configuration format. Please refer to the PROJ documentation.

The configured path can be absolute or relative to the mapproxy.yaml.

Further investigations:

Further investigations seems to suggest that the proj_data_dir option does work but will only consume projections from the proj.db file. All other files alongside proj.db seem to be ignored, but this seems to be because MapProxy won't allow srs authorities other that EPSG.

It looks like MapProxy originally specified using the "epsg" filename for custom projections as the filename must be used as the srs authority by pyproj. However it seems that pyproj won't serve up EPSG codes that come from anywhere but its proj.db file now.

As a work-around I was hoping to use a non-EPSG authority to serve up my custom projection, dropping in an "xxxx" file so I could use an srs of 'XXXX:10000'. Unfortunately MapProxy seems to take any srs provided in the mapproxy.yaml file and change it to 'EPSG'. I tried to use 'XXXX:10000' but MapProxy automatically changed it to 'EPSG:10000', going by the error message.

Seems it will also do this with existing non-EPSG authority projections, such as ESRI, that are held within proj.db and try to use an EPSG equivalent instead. For example MapProxy can't use the existing 'ESRI:102024' projection as it tries to request 'EPSG:102024' instead and so gets the following error pyproj.exceptions.CRSError: Invalid projection: epsg:102024: (Internal Proj Error: proj_create: crs not found).

So the only work-around I can think of is a horrible hack to try and work out how to convert my projection string into the equivalent table entries in proj.db. Unless anyone has an alternative approach I could use?

8
  • If you have PROJ installed already, proj_data_dir is supposed to be the directory where its data is (read up on the PROJ documentation). Once you have identified that, copy an existing one. Feb 12, 2021 at 1:14
  • @alphabetasoup I tried to copy the contents from C:\Python\Python39\Lib\site-packages\pyproj\proj_dir\share\proj to my epsg directory, alongside my custom projection epsg file, but that didn't work. I also copied my custom epsg file to C:\Python\Python39\Lib\site-packages\pyproj\proj_dir\share\proj to see if it would get picked up from there, but no luck that way either.
    – Gavin
    Feb 12, 2021 at 1:44
  • Does the file you added to the PROJ data dir match the existing files in terms of structure, including file name? Feb 12, 2021 at 1:56
  • @alphabetasoup I also tried the same process with C:\OSGeo4W64\share\proj setting the proj_data_dir: 'C:\OSGeo4W64\share\proj\' and copying my custom file across but I now a pyproj.exceptions.CRSError: Invalid projection: epsg:4326: (Internal Proj Error: proj_create: cannot build geodeticCRS 4326: SQLite error on SELECT ... no such table: usage) (truncated error to fit in comments).
    – Gavin
    Feb 12, 2021 at 1:59
  • The structure follows convention, as can be seen from the original post. But file name is different as it's a new file. I have also tried adding the custom projection to other.extra file and the world file, but it still wasn't picked up from those either either.
    – Gavin
    Feb 12, 2021 at 2:02

1 Answer 1

1

The only solution I found so far is to edit proj.db directly to add the custom projections.

It looks like the pyproj library has evolved over time but MapProxy hasn't caught up with its changed file conventions yet.

I've submitted a bug report to MapProxy - https://github.com/mapproxy/mapproxy/issues/493

The easiest way I found to add a new projection to proj.db was to open it up in some SQLite tool and add a new entry to the projected_crs table providing a WKT definition in the text_definition column.

Example SQL insert statement:

INSERT INTO projected_crs (
  auth_name,
  code,
  name,
  description,
  coordinate_system_auth_name,
  coordinate_system_code,
  geodetic_crs_auth_name,
  geodetic_crs_code,
  conversion_auth_name,
  conversion_code,
  text_definition,
  deprecated
)
VALUES (
  'EPSG',
  '100000',
  'CUSTOM ENTRY 1',
  NULL,
  NULL,
  NULL,
  'EPSG',
  '4326',
  NULL,
  NULL,
  'PROJCS["CUSTOM ENTRY 1",GEOGCS["GCS_WGS_1984",DATUM["WGS_1984",SPHEROID["WGS_1984",6378137,298.257223563]],PRIMEM["Greenwich",0],UNIT["Degree",0.017453292519943295]],PROJECTION["Lambert_Conformal_Conic_2SP"],PARAMETER["False_Easting",0],PARAMETER["False_Northing",0],PARAMETER["Central_Meridian",25],PARAMETER["Standard_Parallel_1",-25],PARAMETER["Standard_Parallel_2",-32],PARAMETER["Latitude_Of_Origin",-28.5],PARAMETER["Longitude_Of_Origin",24.5],UNIT["Meter",1],AUTHORITY["EPSG","100000"]]',
  '0'
)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.