2

I'm attempting to build a Docker container (based on an image that includes an installation of GRASS 7.6) which includes a mounted directory into which the outputs of analysis are stored for later use. The intention is to encapsulate some model in a reproducible, sharable, isolated, host-independent environment. The only external link I want is a location of the mapset for reading and writing data: i.e. a persistent store, the actual location of which should be controlled by a Docker volume. I have tried several ways or producing such an environment, but I keep running into permissions errors when either creating or attempting to write to this persistent store.

Say I have a docker-compose.yml:

version: '3'
services:
  grass-service:
    image: grass7
    build: .
    volumes:
      - ./sample-data:/data:rw # This will be the persistent store
      - ./src:/src:ro # Contains code to run analysis
      - ./input:/input # Contains some .tif images to import
    environment:
      - STAGE=dev
      - GISDBASE=/data/grassdata
      - LOCATION=2193
      - MAPSET=project
      - LC_ALL=C.UTF-8
      - LANG=C.UTF-8
    network_mode: host
    stdin_open: true
    tty: true

A Dockerfile built on mundialgis/grass-gis-stable (currently based on Ubuntu 18):

FROM mundialis/grass-gis-stable
USER root
# Install ca-certificates to be able to do TLS (i.e. download extensions)
RUN apt-get -y install ca-certificates

# Set the locale
RUN apt-get -y install locales
RUN sed -i -e 's/# en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/' /etc/locale.gen && locale-gen
ENV LANG en_US.UTF-8
ENV LANGUAGE en_US:en
ENV LC_ALL en_US.UTF-8

ENTRYPOINT ["/bin/bash"]

(Probably ignore the locale stuff... earlier GRASS was complaining about this not being set.)

At this point, there's essentially three related things I want to be able do.

  1. Run a bash script to create/set the GRASS database as some directory inside /data (which is a mount back to ./sample-data on the host).
  2. Download and install extensions to GRASS, ideally at build time so this is only done once. However, if that's not possible (e.g. if they depend on a as-yet-non-existent GRASS DB) then attempting installation at runtime is acceptable.
  3. From here, I'd like to just be able to issue arbitrary grass --exec {command or script} GRASS commands, without an interactive GRASS session, including importing the data from /input with r.external.

One of my difficulties is general unfamiliarity with GRASS, and a particular confusion about how it stores data in "mapsets" and "locations", how it defines computational regions, etc. Compounding this, and the subject of the question, is that I'm unsure how to do this in a way that provides adequate permissions for an interactive Docker session to be able to write output to a mounted, persistent mapset.

I have tried to create the /data/grassdata/2193 location with a mkdir -p /data/grassdata/2193 within my Dockerfile. i.e. RUN mkdir -p /data/grassdata/2193 just before the ENTRYPOINT. This is wrong since the mount doesn't exist at build, but rather is mounted in when the container is run—the other clue being that the environment variables that define the name of the location/mapset also don't exist at build time.

So, instead I made a bash script that makes this directory, and attempts to initialise GRASS within it (/src/init.sh):

mkdir -p ${GISDBASE}
grass76 -text -c EPSG:2193 ${GISDBASE}/${LOCATION}

I first docker-compose -f docker-compose.yml run grass-service (bash entrypoint) and then run /src/init.sh to run my script. This command works insofar as getting the GRASS database established and made persistent... however it also launches me into an interactive GRASS session, which I don't want. So I use the -e flag to start GRASS and automatically terminate it (just a slight modification to init.sh): grass76 -text -c EPSG:2193 -e ${GISDBASE}/${LOCATION}. At this point I run into permissions issues:

Starting GRASS GIS...
Creating new GRASS GIS location <2193>...
ERROR: MAPSET PERMANENT - permission denied
Cleaning up temporary files...
ERROR: MAPSET PERMANENT - permission denied

Additionally, if I try to start an interactive GRASS session at this point, I get booted out after hitting enter, and the following error:

Starting GRASS GIS...
WARNING: It appears that the X Windows system is not active.
A graphical based user interface is not supported.
(DISPLAY variable is not set.)
Switching to text based interface mode.

Hit RETURN to continue.


ERROR: Path '/data/<UNKNOWN>/<UNKNOWN>' doesn't exist
Exiting...

However, the mapset is otherwise successfully created within the mounted volume, and as desired, there is no interactive session. From the host, I can see /test-data/grassdata/2193/PERMANENT and the defaults contained therein (e.g. DEFAULT_WIND). However, due to the permissions issues, I seem unable to install my extensions or otherwise run GRASS commands: grass76 -text ${GISDBASE}/${LOCATION}/PERMANENT --exec g.extension extension=r.hazard.flood gives:

Starting GRASS GIS...
ERROR: MAPSET PERMANENT - permission denied
Executing <g.extension extension=r.hazard.flood> ...
ERROR: MAPSET PERMANENT - permission denied
Execution of <g.extension extension=r.hazard.flood> finished.
Cleaning up temporary files...
ERROR: MAPSET PERMANENT - permission denied

At this point I'm quite stuck. How can I set-up my GRASS environment with a persistent (mounted) store to run GRASS commands from bash, without these permissions errors on the volume?

Note, I suspect it's relevant that my volume also happens to be a CIFS share. I will experiment momentarily and if the result is different when using a truly local store, I will make a relevant edit. However, I currently discount the possibility, since at least one of the above experiments shows it is possible to write to this mounted volume from inside the container.

  • Hi, did you look at grass.osgeo.org/download/software/docker-images ? I have re-done most of the docker images over the last days. The dockerfiles are also there (following the links in hub.docker.com to the respective github repos). – markusN Aug 15 at 12:11
  • Yes, but I'm still getting into difficulty when trying to use them as part of a docker-compose workflow that includes external mounts. I've solved some of my issues and will attempt a complete answer when I've solved the remaining ones. In particular, at the moment I want to set my default region to the extent of an external raster dataset. However because the default data is missing from the PERMANENT mapset, I have issues. I realise these are generic GRASS issues; but if you have a complete example of your Docker image, with docker-compose and an external mount, I'd love to see it @markusN – alphabetasoup Aug 19 at 3:56
1

I have managed to produce a reproducible environment that, although it's not my perfect solution, at least gives me the primary things I want, with only a few limitations.

Objectives

  • Uses Docker Compose
  • Has permission to write to a mounted volume used as a GRASS database.
  • Has permission to read from a mount over ssh
  • Can install extensions persistently

Not all of these are directly relevant to the question, I just mention them for context.

Limitations

  • Requires mounted source code that needs to run on initialisation, i.e. after and independently of the Docker build
  • Messy, and probably manifests some other issues that are not apparent for my use case

Solution

I have a very simple Dockerfile that just allows me to get into an interactive session from Docker Compose. The intention is to run bash scripts via Make (semi-interactively), with read-only input data, writing output to a persistent store, before exiting the image.

Dockerfile:

FROM neteler/grassgis7:latest
USER grass
CMD ["/bin/bash"]

docker-compose.yml:

version: '3'
services:
  grass-service:
    image: grass7
    build: .
    volumes:
      - ./data:/data:rw
      - ./sample-data:/sample-data:ro
      - elevation-data:/input/dem:ro
      - ./src:/src:ro
    environment:
      - STAGE=dev
      - HOME=/data/
      - GISDBASE=/data/grassdata
      - LOCATION=nz
      - LC_ALL=C.UTF-8
      - LANG=C.UTF-8
    network_mode: host
    stdin_open: true
    tty: true
    user: "1001:1001"
volumes:
  elevation-data:
    driver: vieux/sshfs:latest
    driver_opts:
      sshcmd: "${sshuser}@${host}:/path/to/some/data/over/ssh"
      allow_other: ""
      password: ${sshpassword}

The user part is important.

In ./src I have an initialisation script, init.sh (which primarily installs extensions) and then a sample Bash script for running GRASS commands, test-grass.sh; both demonstrate how I use GRASS from within my container.

init.sh:

#!/bin/bash

mkdir -p $GISDBASE
# Initialise a GRASS Location so there is a default region
grass -text -c EPSG:2193 -e $GISDBASE/$LOCATION
# Add GRASS sample data as well
cp -r /sample-data/nc_basic_spm_grass7 $GISDBASE/nc_basic_spm_grass7

# Define an alias to start GRASS with a particular Location and the PERMANENT Mapset
GRASSSTART="grass -text $GISDBASE/$LOCATION/PERMANENT --exec"

# Load extensions
# https://grass.osgeo.org/grass76/manuals/addons/r.hazard.flood.html
$GRASSSTART g.extension extension=r.hazard.flood
# https://grass.osgeo.org/grass76/manuals/addons/r.stream.distance.html
$GRASSSTART g.extension extension=r.stream.distance

test-grass.sh:

#!/bin/bash
# Copies some data to test with, and writes a TIFF output, handling a datum issue in-between

LOC=nzdem
GRASSTART="grass -text $GISDBASE/$LOC/PERMANENT --exec"

# https://lists.osgeo.org/pipermail/grass-dev/2015-April/074834.html
# Import external data (NZ 15m DEM)
grass -text $GISDBASE/$LOCATION/PERMANENT --exec r.in.gdal input=/input/dem/nzdem.img output=nzdem15 location=nzdem
# Set computational region to be equal to this raster dataset
# GRASS doesn't recognise its datum (NZGD2000_NTv2) on import: so set it explicitly to prevent it defaulting to WGS84
$GRASSTART g.proj datum=nzgd2k -c
$GRASSTART g.region raster=nzdem15 -p
# Test a command (write out as a tiled GeoTIFF)
$GRASSTART r.out.gdal -f input=nzdem15 output=test.tif format=GTiff type=Int16 createopt='COMPRESS=DEFLATE,BIGTIFF=YES,TFW=YES,PROFILE=GeoTIFF,TILED=YES'

I can't do these on build, even the extension installation, since they must be installed within a defined Location with a PERMANENT Mapset. (This is the confusing part of GRASS that I dislike.) Given I want my GRASS data to persist on a mounted volume, I need to do this after build, and for each volume (in my case at least twice, according to the STAGE I'm working in).

Do put it all together, I do: docker-compose -f docker-compose.yml build grass-service then docker-compose -f docker-compose.yml run grass-service. This places me into an interactive Bash terminal. From there I run /src/init.sh (then /src/test-grass.sh, etc.).

This process is much messier than I'd like, so I'll leave the question open for a better, cleaner solution. But this at least gives me reproducible access to all the excellent GRASS processing algorithms, without a graphical session.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.