What is the most appropriate technology for publishing Census (and other) data for small areas?

Question

We have a web application which shows the (United Kingdom) Census data on thematic maps using client-side code (based on OpenLayers) and KML.

// This section can be skipped

// begin

At the moment it works in the following way

• We store area shapes as KML in database (just shapes, no styles, colours or other data)

• We also store the Census data against each area (e.g. area="Output area E01000123", time period=2011, variable= "% of single people", metric value = 34.12) in the same DB. Of course, data structure is normalized.

• A user selects a region (e.g. London) and a variable

• We get all the small areas within the selected region, generate the KML and apply colors based on metric values.

• The web page displays KML on a map (using OpenLayers-based Javascript code).

//end

This approach works well when the number of small areas in a region is not too large. In some cases the number of small areas might reach 5000. In this case client-side code is too slow in IE8 (which we have to support). So I guess we have to generate an image from the shapes server-side.

What is the most appropriate technology?

In the past we used GeoServer with PostGis backend. It was not too fast and quite unstable. I can explain how we used it.

UPDATE

I decided to use GeoServer + PostGis. GeoServer serves images. When a user selects an area, the client-side code creates a custom SLD and passes it to GeoServer. When no or few areas are selected, it is very fast (~1sec), even with 30000 polygons in the layer. If a user selects e.g. 2000 areas, it takes several seconds (I guess because GeoServer has to process a huge SLD file).

Answer 1

Yes, tiling is the right solution to increase performance, but you will need more disk space.

In the Open-Source, Mapnik + TileCache or TileStache are the best libraries (in my opinion) to generate tiles. You can serve your tiles with GeoServer, MapServer or GeoDjango.

If you dont want to tile, remember that 5000 features its quite a lot to display on a single map, but it always depend on the complexity of your features. Markers could be clustered for instance, but complex polygon would always be slow. You can tune your DB by creating spatial indexes to improve performance, simplify your features or display only a part of the dataset on map.

Answer 2

I'd strongly recommend that you store data as geometries in whatever database you want to use. Storing KML sounds inefficient.

There are a number of products that you could use. I recommend you look at MapDotNet (www.mapdotnet.com). 60 day trial is free, Express (thread limit) is free. See also www.mapsjs.com for examples of HTML5 clients.

Disclaimer: I am one of the developers of MapDotNet.