Advice on benchmarking my Tile Server (Apache2+mod_wsgi)

Question

I'm wondering what advice people have for benchmarking a tile server?

Specifically, I'm running a TMS using TileLite, and Vector Tiles using TileStache. Both are running under Ubuntu Server 10.10 / Apache2 / mod_wsgi, using different virtual hosts. Both are pulling data stored in PostGIS, and are caching to disk. I'm displaying these tiles using Polymaps.

I'm interested in establishing a benchmark for TileLite and TileStache, then exploring options for increasing their performance/reliability.

I've tried out using ab on static (individual tiles), but I'm unsure how to simulate a user interacting with the map, causing numerous tiles to be rendered.

Anyways, any tips or sharing of experiences that people have had would be greatly appreciated!

thanks, matt

N.B. Apparently I can't create new tags like 'TMS' or 'Apache2' because I'm too new to the forum :(

Answer 1

As tile caches are pregenerated map images, you will really be testing the underlying map / wms / image server. Unless you are testing against a pregenarated cache, in which case you are testing the web server itself (Apache / IIS etc.).

Stress testing for websites is often done using JMeter. There was a recent series of posts on using JMeter at http://www.sourcepole.ch/jmeter-series

JMeter was used to benchmark various WMS servers at FOSS4G - http://wiki.osgeo.org/wiki/FOSS4G_Benchmark - which will be a similar process to benchmarking a TMS server.

Sample code and scripts for each of the WMS servers specified geographic areas can be found at http://svn.osgeo.org/osgeo/foss4g/benchmarking/ Further discussopns on this benchmarking are in the discussion archives.