Using Node.js to cluster millions of points on the server side

Question

Situation:

I am testing out different visualization techniques for web mapping a data set that has about 2.5 million points and a lengthy set of attributes (almost 30 fields). So its quite a large file. I want to try clustering this data. I know sending that much data to a client will either result in a time-out or crash so I've been look at other methods and found the supercluster with mapbox. I used the top rated answer in this question to build a web map with Leaflet that handles about 500 000 points pretty well (approx 6 seconds to load). While this is good its only a subset of the data and its not as instantaneous as the Supercluster claims. Now I want to test using Node.js to do the clustering serverside, this way I can hopefully load the full 2.5 million point data set quickly.

Question(s):

I've never used Node.js before but I'm getting the gist of it. I need some help figuring out how to use the supercluster with Node.js because the GitHub site doesn't have clear instructions for beginners. I'm at the stage where I'm using the Demo index.html and its not showing me any points (I have not changed anything in the scripts).

Has anyone used the supercluster with Node before or has any general tips and tricks for me?

Also can I use GeoServer with Node.js?

Or if someone has a different suggestion for clustering 2.5 million points on the fly I'm interested in any and all options!

Specs:

OS -> Windows 10, Node.js -> 12.14.1, Leaflet -> 1.3.1

Answer 1

Shortly:

1. Firstly, define your supercluster and load with geojson.

   const Supercluster = require('supercluster');
   const index = new Supercluster({
     radius: 40,
     maxZoom: 17
   });
   index.load(geojson);
   

2. Get map zoom and map bounds from frontend. Than, pass it in index to get data.

   let clusterMarkers = index.getClusters(box.bounds, box.zoom);

3. Then, just send clusterMarkers to front. That's it. For example:

   socket.emit('get_clusters', clusterMarkers);

4. If you want to get data when user click the cluster you have to send getClusterExpansionZoom parameter to supercluster ( instead of box bounds).

BTF, the demo of supercluster is written very clear. Try to get concept.