7

The Problem:

Using OSM-Bright in combination with an imported (osm2psql) Europe shapefile, tiles won't load unless I have 2-3 layers active (of the total of 29). I don't have this problem with an imported Netherlands or Amsterdam shapefile. So I'm assuming it has something to do with the sheer amount of data that gets returned by the queries?

My Setup:

I am running TileMill with OSM-Bright on an Amazon EC2 c3.2xlarge:

8 x Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz 15.8GB RAM / linux ubuntu 14.04 trusty / TileMill 0.10.2

The only changed settings of Postgres I have done are:

work_mem = 2MB maintenance_work_mem = 32MB

Detailed description & what I have tried:

When loading the map, it seems as though the query times out. I have tried making a swapfile and monitoring memory when opening the project. The swapfile doesn't get touched and the memory gets drained to ~1% and stays there.

I won't get any errors when it 'times out' (tiles get the 'image not available' icon), not in the client interface, and when running TileMill via SSH there are no errors in the output. Very rarely I get a "PostGIS connection: null" error in the TileMill interface. When this happens, I get { [Error: socket hang up] code: 'ECONNRESET' } in the Tilemill log.

Screenshot: empty tiles

Is there anything I can do to optimize the loading of the queries so I can work with the huge Europe dataset? Or is there any good practice in handling such large datasets with TileMill & OSM-Bright?

PS. for reference, this is the script I used while importing Europe into postgres (I don't know if it is relevant):

sudo osm2pgsql -c -G -C 14000 --cache-strategy sparse -U postgres -d osm /data/downloads/europe-shapefile/europe-latest.osm.pbf

Improve this question

1 Answer 1

Reset to default
4
+50

I ran into the same problem and I ended up using adopted way_area size clauses. By default they seemed to be too small for me. So when zooming out it would fetch so many very small polygons that that was too much data to transfer. So basically you don't display too small polygons when being zoomed out. You have to try and error until you have the best results for your need.

Next thing I did was to simplify polygons. When you zoom out, f.e. the border of a country doesn't need to be that detailed. You can use Postgis (ST_simplifiy) for that task. That again helps to keep transferred datasizes smaller (a lot smaller).

Those two things helped A LOT getting my map to work again with a huge dataset.

I'd be interested to know what else you did to improve the rendering performance of your map?

Improve this answer
15
  • Awesome, thanks for helping me! Glad to see that I'm not the only one having this problem. Could you give an example how you wrote the way_area size clauses? I don't fully understand the imported table columns yet, what unit are the way_area sizes? Is that square meters or anything similar?
    – Thermometer
    Commented Nov 10, 2014 at 8:27
  • I am not sure which unit they are in... I found that: For "geometry" type area is in SRID units. For "geography" area is in square meters.... But what I did (I am a non GIS guy), I searched for something I know how big it was (Like a lake... compare to Wikipedia) and see how big the way_area is. Then start adopting your way_area clauses... Hope it helps...
    – Georg
    Commented Nov 10, 2014 at 12:10
  • Ah that is a good idea to get a relative sense of what the value means. I will fiddle with it to optimize it. Could you give an example of a query where you simplify the geometry? I wrapped ST_Simplify around way in my query: SELECT ST_Simplify(way), way_area AS area, COALESCE(landuse, leisure, "natural", highway, amenity, tourism) AS type but I get the Postgis Plugin: ERROR: function st_simplify(geometry) does not exist error. Thanks for helping again, I am a mobile developer by trade and a bit new to GIS also.
    – Thermometer
    Commented Nov 10, 2014 at 14:58
  • write SELECT ST_Simplify(way) as way so the as way is missing :)
    – Georg
    Commented Nov 10, 2014 at 15:25
  • 1
    Look at your postgres DB processes. See what's taking so long... If it's the ST_Simplify(way) you might want to pre-calculate them... Or if it's not the calculation of ST_Simplify(way) but the data transfered, then try to use more aggressive simplification... When zoomed out to 0-6 the shapes can be super simplified i guess... Landuse and water are separate layers in my map yes. But that was the default of OSM_Bright?!?
    – Georg
    Commented Nov 11, 2014 at 15:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.