35

When using Leaflet to visualize a large dataset (GeoJSON with 10,000 point features), not surprisingly the browser crashes or hangs. A sub-sample of 1000 features from the same dataset works flawlessly. Unfortunately I can't share the dataset for others to try out.

Does anybody have better solutions for visualizing such large datasets? (ultimate aim is to scale this to 2 million features) I'm even willing to consider offline visualization frameworks in case browser based alternatives such as Polymaps or d3.js etc. are deemed incapable.

Edit: Forgot to mention, the user needs to be able to filter the dataset by attributes. So out of N features, only the matching n <= N features might need to be dynamically rendered.

23

I'm Leaflet author. There's an awesome clustering plugin for this, Leaflet.markercluster. It's very fast and efficient (take a look at 50k markers example), looks and works very smoothly with nice animations, and has lots of options to suit to your needs.

16

You can use TileMill and render points as raster images, with fast interactivity from UTFGrid. It scales to millions of points and polygons, like this census map, since it intelligently sends only the data needed for specific areas, exactly when it's needed.

As far as I know, there are no other fast methods for doing this other than having a very fast WFS server, which is rather hard to maintain/scale to many viewers.

Disclosure: work for MapBox, wrote quite a bit of the code. But TileMill is free/open source, etc.

  • 1
    I forgot to mention the user should be able to filter the dataset by attributes and display only matching records. So let's say out of 10,000 records, only 500 may actually need to be rendered for a given case. Can (or how would) I do this with TileMill? – Imad Oct 1 '12 at 9:16
  • 3
    Nope. You might want to try CartoDB, but you should know that making things dynamic and making things performant are opposing goals. – tmcw Oct 1 '12 at 14:31
  • 1
    the census map link is dead! – drho Nov 22 '18 at 15:20
  • All the links are daed. – LeeGee Jan 25 at 3:41
9

Have you looked into the leaflet clusterer? A blog post by the author describes it here

Another option worth a look may be to use leaflet in combination with GIS Cloud. Take a look at this demo to see it handle a lot of geometries very quickly. Very impressive. I am in no way affiliated with GISCloud.

7

You should never display millions of points on a map. Not only because of the major performance problems, but also from a user's perspective because for them it most certainly will be difficult to interpret this data. Use some means of aggregating the data (clustering, aggregating to polygon areas etc.) combined with different display types at different zoom levels (e.g. show the "raw" point data only on very high zoom levels and use aggregated data everywhere else). An example would be a real estate site such as zillow.com.

  • 8
    You should never say "you should never". Locals & Tourists is a great example of the kind of insight that visualizing millions (or billions in this case) of points can give. – Joseph Sheedy Mar 13 '15 at 18:30
  • 1
    Agree w/ @velotron, e.g. six million points beautifully and quickly rendered: mapbox.com/blog/supercluster – Max von Hippel Oct 28 '16 at 4:07
  • 1
    Yeah, but in this case those points are clustered as well (depending on zoom level), making it very easy to interpret and understand the data. – chriserik Oct 30 '16 at 10:22
  • 2
    @chriserik right, so the correct answer would be "you should use clustering or heat maps if you have that many points, and here is how to do that ..." – Max von Hippel Nov 2 '16 at 8:35
  • 3
    My point was that unclustered plots of millions of points can provide insight. – Joseph Sheedy Nov 15 '16 at 0:45
0

I suggest you reduce the rendered amount of point features: The human eye won't be able to see 10,000 points, not to speak of 2,000,000.

What you could try is dynamically requesting the dataset from a custom server (which you'd have to set up), e.g.

    map = ...
    map.on('moveend', function(e) {
        getGeoJson(e);
    });
    map.on('zoomend', function(e) {
        getGeoJson(e);
    });
    map.setView([2,3], 2);

    function getGeoJson(event) {
        // todo determine current viewport
        $http.get('someGeoJsonDataProvider.someLanguage?currentView=[lat0,lon0,lat1,lon1]').then(function (resp) {
            // todo clear layers
            // new layer
            map.addLayer(
                L.geoJson(resp.data)
            );
        });
    }

Your server would then calculate which points to return, depending on what the user wants to see right now. Depending on zoom factor and clipping, you will only need to return a very small percentage of points without worsening user experience.

Downsides: Setting up a server (you should find libraries for filtering the geo points) & slower rendering (after every zoom or drag, a server request needs to be made)

-5

I had solutions to map out 50 to 100 million records, you need to use server-side solutions to do grid and dynamic based. You can not reply on web map APIs (Google,or others) to do client-side's rendering....

[http://96.231.36.9:8080/rbgis/google_map.html][1] try the above links and see how

  • 2
    Please expand your answer, so it will be helpful even when your server is inaccessible. – lynxlynxlynx Sep 4 '13 at 15:28
  • Yeah that's ironic. Since your server-side link doesn't work. – Max von Hippel Oct 25 '16 at 6:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.