I'm working on a project, where I have to visualise a lot of lines (and dots) on a map. We used to use openlayers, our own opensteetmaps tile server for "background" maps, and then generate a GeoJSON with 1-5k vectors and points in different colors, and it kinda worked fast enough in the browser (first 'render' took some time, but when zoomed in it wasnt too slow on a fast computer). It's hard to explain what exactly it showed, but imagine 'acceleration vectors' of a car driving down the road - so a point where the car was at the moment, and a line showing direction and acceleration force. Each car had a different vector/point color.
Then, things changed, and 1-5k became 1-5 million points+lines, and that number will probably still grow, GeoJSONs are huge, and browsers just comes to a halt at 100% CPU usage.
So, how to draw them? I used to do something similar with points in php+gd and overlaying a google map, but it didn't show a vector if both (start, end) points were outside the visible field (but the vector crossed it). There could be better options for just that, but we don't know any.
The second option was making a tile server with transparent tiles and overlaying it over the osm maps, but except for following the tutorial for open street maps, we've never done that, and we don't know if it's a good/optimal idea.
I'd be also happy with any other ideas.
Our data is currently in a mysql database with DOUBLE lat/lon columns, but we have no issues with moving it to a postgis database.