Density Mapping with mapnik

Question

I have a database with millions of points so I'm aggregating data returned from the source in the form of (lat,lon, total_count). Where lat/lon is the centroid of a geohash and total_count is the total number of records that fall within that geohash.

I'd like to basically create density map from these aggregations and color code by density. (Ie...The higher values will be darker colors).

Essentially I'd like to create a density heat map using mapnik. I've tried colorize-alpha, however, it doesn't seem to consider "total_count" or weighted aggregations. Colorize-alpha would seem to work great if all points were considered equal.

However, In my case since I'm aggregating with each bucket having a different density, that is not the case...and I haven't been able to find a way using colorize-alpha to work.

I'd like to do something similar to what I see here (On the left for mapnik).

http://cartodb.github.io/VECNIK/examples/density.html

I was able to write a style that can color code based on total_count, however, when points overlap it doesn't look so good. Maybe my composting is not ideal but I'm not real sure what I should set that to.

What would be the best strategy to "merge/blend" colors and sort of smooth things out.

Basically, colors on top of each other should get a little darker (if 2 yellow's are on top of each other then it should turn a little darker shade).

If I could style like the website above I would be happy. Any thoughts on how to get a good visualization using mapnik. (Node-mapnik).

Where typically even with aggregations I could around a million records so I need quick symbolization. (Like the DotSymbolizer gives me).

Basically looking for a density type heatmap that considers weights/total_counts.

Answer 1

Way late to the party here, but if I were in your situation, I'd create a new table in the database and write a script to iterate over your existing points, and for each point, create N representative records corresponding to your total_count property for that parent-record.

If you want to take it a small step farther, for each child-record, I'd offset its lat/lon values a statistically-insignificant distance on both lat and lon so any software that touches it (i.e. Mapnik, etc.) will see a truly unique point. I'm not sure how granular or "zoomed-in" your spatial data is, but if it's more zoomed-out than "house-level", then you can safely mess with the 5th and 6th decimal positions without polluting the data. However, if you are zoomed in quite close, you should research the appropriate offset distances.

But my suspicion is you don't need to randomize the offsets, you can probably just breakout the child records and go from there.