Automating NetCDF data update on GeoServer

Question

I have published a NetCDF file (received on 2018-01-02, hourly data over the following 7 days) to GeoServer as WMS with Time enabled.

I was able to preview the layer with OpenLayers with the "time" attribute attached to the URL to the GetMap command.

http://{GEOSERVER_DOMAIN}:8080/geoserver/fdi/wms?service=WMS&time=2018-01-02T01:00:00.0Z&version=1.1.0&request=GetMap&layers=fdi:GFDI_SFC&styles=&bbox=140.8879867553711,-39.76999832100885,151.05409088134766,-33.929998627233346&width=768&height=441&srs=EPSG:4326&format=application/openlayers#toggle

The project I am working on requires the NetCDF file to be updated on GeoServer on a daily basis.

When I was doing some testing with the 2017-12-28 data (3 days older), I was not able to preview the image for 2017-12-28T01:00:00.0Z

http://{GEOSERVER_DOMAIN}:8080/geoserver/fdi/wms?service=WMS&time=2017-12-28T01:00:00.0Z&version=1.1.0&request=GetMap&layers=fdi:GFDI_SFC&styles=&bbox=140.8879867553711,-39.76999832100885,151.05409088134766,-33.929998627233346&width=768&height=441&srs=EPSG:4326&format=application/openlayers#toggle

I was not able to preview data for any date and time earlier than 2018-01-02T01:00:00.0Z.

I have turned off Tile Caching and did disable and enable the NetCDF data store.

Is there anything else I need to look at/adjust to get this working? Is this a good practice to update the data source on a regular basis on GeoServer.

Edit - time dimension from the GetCapabilities request

<Dimension name="time" default="2017-12-27T18:00:00Z" units="ISO8601">
2017-12-27T18:00:00.000Z/2017-12-27T18:00:00.000Z/PT1S,2017-12-27T19:00:00.000Z/2017-12-27T19:00:00.000Z/PT1S,2017-12-27T20:00:00.000Z/2017-12-27T20:00:00.000Z/PT1S,2017-12-27T21:00:00.000Z/2017-12-27T21:00:00.000Z/PT1S,2017-12-27T22:00:00.000Z/2017-12-27T22:00:00.000Z/PT1S,2017-12-27T23:00:00.000Z/2017-12-27T23:00:00.000Z/PT1S,2017-12-28T00:00:00.000Z/2017-12-28T00:00:00.000Z/PT1S,2017-12-28T01:00:00.000Z/2017-12-28T01:00:00.000Z/PT1S,2017-12-28T02:00:00.000Z/2017-12-28T02:00:00.000Z/PT1S,2017-12-28T03:00:00.000Z/2017-12-28T03:00:00.000Z/PT1S,2017-12-28T04:00:00.000Z/2017-12-28T04:00:00.000Z/PT1S,
... ...
</Dimension>

Answer 1

Just replacing the NetCDF file on disk, behind GeoServer's back, will not work in general for a couple of reasons:

• Some information about the file is kept in memory, the new file might have a different set
• To speed up access to the file contents a set of index files is kept on disk, which will become outdated vs the new file. The indexes are kept in a hiddent (starting with a ".") directory on the side of the NetCDF file.

Options:

• Clean up the indexes along with the file and then force GeoServer to flush its cached information by summitting a reset command (e..g, from the UI, the "clear" button in the status page, or using the REST API)
• Use the REST API to remove and re-create the file and associated layers

Answer 2

Here is a script I made to refresh NetCDF coveragestores. It downloads the list of existing coveragestores, parses it using jq (a JSON commandline utility), downloads each one, and re-uploads each one.

srcdir="$(dirname "$0")"
source $srcdir/variables.sh
curl -u $username:$password $geoserver_base_url/geoserver/rest/workspaces/$workspace/coveragestores.json | jq '.coverageStores.coverageStore|.[]|map(.)|.[]'| while read name; read url
do
    name=`echo $name|sed 's!"!!g'`
    url=`echo $url |sed 's!"!!g'`
    coveragestore="${srcdir}/coveragestores/${name}.json"
    curl -u $username:$password $url > $coveragestore
    curl -u $username:$password -XPUT -H "Content-type: application/json" -d @$coveragestore $url
done

variables.sh just contains things like $username, $password, $workspace and $geoserver_base_url (pretty self-explanatory variables).