Batch process for seeding multiple BBox areas in Geoserver

Question

I am using GeoServer v2.8 and v2.9 with integrated GeoWebcache. I am seeding tiles over a very large area and obviously don't want to seed down to z10+ in un-populated areas or the sea as this is a waste of cache space and seeding time. What I would like to do is determine a whole set of BBox areas to seed at different zoom levels. Determining the BBox areas is a straight-forward GIS task. I can manually enter each one in the GWC seed form and watch it until it finishes and then enter the next one. However, it would be more efficient if there were a utility (or can anybody describe a method) for queuing BBox areas and Z seed levels so that, when one BBox seeding process has finished, the next will start the next one automatically.

Perhaps this could be done via the REST interface but I'd need some sort of trigger back from Geoserver to say when each BBox areas is done, to know when the next task should be sent. Any ideas?

Answer 1

GeoWebCache does not have queuing system for seeding processes. Therefore you can't just send hundreds of seeding requests with REST API because that would start them all at the same time. However, the REST API does return the status of running processes and it should be possible to ping the running processes and start new ones once the previous ones have been finished but I have never tried that. I simply create seeding processes of equal size and measure how long they usually take. If seeding is usually finished in 15 minutes I make a script that starts new processes after every 20 minutes. It is not optimal because usually this leads to 5 minutes idle time between the processes. What is more important for me is that by keeping the seeding processes small I can be rather sure that they are successful.

One big problem with seeding the GWC cache is that user does not see any difference between success and failure. If some seeding process fails it just disappears from the list and the same happens when the seeding gets finished fine. Therefore I tend to give 4-6 threads for each seeding process and keep the processes relatively small. This way some threads may die but the seeding process can still continue to the end with the remaining threads.

REST seeding commands should look about like these ones:

curl -x cache.com:8080 -v -k -u admin:geoserver -XPOST -H "Content-type: application/json" -d "{'seedRequest':{'name':'GWC_layer','bounds':{'coords':{ 'double':['1.0','1.0','2.0','2.0']}},'srs':{'number':4326},'zoomStart':8,'zoomStop':11,'format':'image\/png','type':'seed','threadCount':4}}}" "http://gshost:8080/geoserver/gwc/rest/seed/GWC_layer.json" 

Wait x minutes, give a new BBOX

curl -x cache.com:8080 -v -k -u admin:geoserver -XPOST -H "Content-type: application/json" -d "{'seedRequest':{'name':'GWC_layer','bounds':{'coords':{ 'double':['1.0','2.0','2.0','3.0']}},'srs':{'number':4326},'zoomStart':8,'zoomStop':11,'format':'image\/png','type':'seed','threadCount':4}}}" "http://gshost:8080/geoserver/gwc/rest/seed/GWC_layer.json"

Your idea about getting something back from GeoServer/GWC side for triggering the next seeding reques (or re-seeding request in case of interrupted process) is excellent. I recommend to read https://github.com/geoserver/geoserver/wiki/Successfully-requesting-and-integrating-new-features-and-improvements-in-GeoServer and follow the advice.