0

I am running GeoServer 2.8 on Ubuntu 14 as a WAR in Tomcat 7. I have an 8 core machine with 16GB of RAM and have allowed a generous Heap size of 4GB. I have JAI installed and enabled. Meta tiling is set to 8x4 and I have allowed 3GB for rendering so memory should not be a problem.

I have a layer-group with scale-dependent styles. The various layers transition from simple global data to OSM. The problem is that at the global or low-zoom levels, Geoserver seems to be attempting to load and process all data in all layers even though most will not be rendered. When you consider OSM - that is a LOT of data! I have tried to ease the pain by making views of low-zoom data and high-zoom data (e.g. for roads the low-zoom misses out a lot of the detail and minor roads). This does not help and possibly compounds the issue. Looking at memory consumption as a whole, HTOP shows that Java is barely using any of the 4GB heap allowed. So throwing memory at GeoServer doesn't help.

The irony is that rendering is fine at mid to high zoom levels, even though now the OSM data is visible, presumably as the data requested is much more limited. All my OSM layers are spatially indexed and clustered - so no problem here. Even so, that should be irrelevant at low zoom levels as the OSM layers are not visible in my styles.

At the moment I am working around the problem by setting my render times for WMS to 1 hr and seeding the cache (very slowly - until I get to intermediate zoom levels when the data load reduces to sane amounts). This is not a great solution, especially if I make a minor change to my layer-group.

I want a single layer-group to keep things simple at the client end. The best solution, as I see it, is to implement a scale-dependent 'switch' in the layer-group definition but that requires GeoServer-Developer input. However, what practical methods do people suggest to improve performance in the meantime (I have already read GeoServer on Steroids and implemented everything relevant in there).

EDIT
The problem was due to a corrupted style that cause a large volume of very small polygons to be drawn at all levels - which explains the observed results. fixing the style, fixes the problem. However, this has highlighted what appears to be a serious memory leak following failures, which progressively causes Geoserver to become less responsive and only a restart of Tomcat combined with a nasty killall -9 Java call fixes (just restarting Tomcat leaves the leaks as checking for leaks in Tomcat manager will confirm).

In my logs I periodically have the following error:

ERROR [org.geotools.map] - Call MapContent dispose() to prevent memory leaks

The error is never reported singly , but always multiple times (into double figures sometimes) and is associated with a few DEBUG logs especially:

DEBUG [org.vfny.geoserver.responses.wms.map] - setting to transparent
2015-08-08 08:59:48,448 WARN [org.geotools.renderer.lite.gridcoverage2d] - Could not reduce the grid geometry inside the valid area bounds: ReferencedEnvelope[-1.7976931348623157E308 : 1.7976931348623157E308, -85.0 : 85.0]
Grid geometry isGridGeometry2D[GridEnvelope2D[6365..7293, 7345..7818], 

and:

DEBUG [org.geoserver.security.IncludeQueryStringAntPathRequestMatcher] - Checking match of request : 'Path: /web/, QueryString: x=0bkVEjecmbChX7z8UByFaQCm3CLIUMoGOhTdEK9-vGhb41MvY3jdDsgkHKnbSgYXWRKtVX*xG5ASH1XP*FxePw'; against '/web/**'
2015-08-09 16:39:43,123 DEBUG [org.geoserver.security.IncludeQueryStringAntPathRequestMatcher] - Matched Path: /web/, QueryString: x=0bkVEjecmbChX7z8UByFaQCm3CLIUMoGOhTdEK9-vGhb41MvY3jdDsgkHKnbSgYXWRKtVX*xG5ASH1XP*FxePw with /web/**
2015-08-09 16:39:43,123 TRACE [org.geoserver.ows.OWSHandlerMapping] - No handler mapping found for [/web/]
2015-08-09 16:39:43,123 TRACE [org.geoserver.ows.OWSHandlerMapping] - No handler mapping found for [/web/]
2015-08-09 16:39:43,123 TRACE [org.geoserver.ows.OWSHandlerMapping] - No handler mapping found for [/web/]
2015-08-09 16:39:43,123 TRACE [org.geoserver.ows.OWSHandlerMapping] - No handler mapping found for [/web/]
2015-08-09 16:39:43,126 DEBUG [org.geoserver] - Thread 1587 locking in mode WRITE
2015-08-09 16:39:43,126 DEBUG [org.geoserver] - Thread 1587 got the lock in mode WRITE
2015-08-09 16:39:45,927 ERROR [org.geotools.map] - Call MapContent dispose() to prevent memory leaks

This question can be closed as it has now strayed from GIS into programming and this isn't the forum for it - I'll mark the only answer as the solution.

3

Ciao, a few things:

  • It looks to me you are mostly rendering vector data, hence JAI does not helpat all

  • If you are mostly serving vector data with complex styling rules, I am hoping you are using a DBMS rather than shapefiles especially for OSM. If not at global scale GeoServer will have to load all the shapefile into memory and then filters as we only have spatial indexes on shapefiles

  • if you are using a spatial DBMS, you can use verbose debugging and the SQL that GeoServer is producing so that you can understand if you need to restructure your data and/or making styles simpler

  • Scale based exclusion rules in SLDs are evaluated before loading data so that should not be a problem.

  • If the data you want to use at global scale is too defined (i.e. the geometries are heavy to load and too detailed) I would suggest to simplify them upfront in order to reduce rendering time (although some simplification can be tweaked if you use PGis directly inside the layer definition)

  • One last thing, you are using 2.8 which has not been officially released (although it's closed to being released), hence you might run into bugs :)

Simone.

  • Many thanks for your comments. I had the exact same issue in 2.7 so I don't think that's it. Yes I am using a DBMS (PostGIS). At low zoom levels the data is very simple - just some points and a low-res raster, which by themselves take no more than a minute to render. It must be something to do with the other data in the layer-group. Some of my views are for low-res use. I'll see if verbose debugging throws anything up (although it will become a little like spotting a needle in a haystack given the volume lines already in the logs!). – MappaGnosis Aug 6 '15 at 16:35
  • Some points and a low-res raster can't take that much time to render (broad sentence, but you get the point :) ) I would try to reduce to a minimum number of layers that is slow and then look at the logs. Keep us posted. – simogeo Aug 7 '15 at 7:29
  • Will do! The raster plus points only takes that long when they are part of the big layer-group. In a group by themselves, they don't take long. So, I am absolutely convinced that the issue must be related to the loading of irrelevant data (even though you say scales are polled in styles before loading data). How else can 1 simple tile (@min zoom and only two layers) take an order of magnitude more time than 64 tiles at zoom level 9 (15 layers of OSM views with complex styles, but no raster)? None of the OSM should be loaded for the min zoom levels (0 to 4). – MappaGnosis Aug 7 '15 at 8:13
  • As I said, try to get down to a minimal set of layers that are problematic then change the log level and let's look at that. – simogeo Aug 7 '15 at 13:19
  • Sorry for the slow reply but other work has got in the way. I shall do some systematic tests. The first concern I have though, is that there is a memory leak possibly associated to failed renders. Over time, I notice that it becomes increasingly difficult to connect to Geoserver and that the number of java processes slowly grows. The logs sometimes show warnings relating to memory leaks. I need to restart Tomcat as my server has become unresponsive again, but I shall try to get some more diagnostic soon. – MappaGnosis Aug 10 '15 at 7:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.