0

I'm dealing with around 150,000 points on a map. My customers need to see all the points, wthout clustering. I tried Openlayers and Mapbox GL Js, both have the same issue: its memory costs are huge!

I stabilized on Mapbox GL JS 0.15 (max. 1GB RAM during loading, min 700MB RAM) but since it's not working on Chrome (some tiles are freezing) I have to go with the new version, which is nice but has a memory map (for my case) of 1.6GB!

My biggest customer has a computer with 4GB RAM, this is clearly not working...

I know this is my fault to display all that points on the client side, but I don't have any experience of finding a good & free solution, client side or server side.

Example of my map on Mapbox GL JS 0.26 (loading is long as I load my data from dropbox)

12
  • 2
    Maybe server side rendering via Mapserver, Mapnik etc.? Oct 17, 2016 at 7:59
  • @DmitryBaryshnikov that's what I was thinking about but I'm starting from nothing.. it has to be dynamic, like filters, auto refresh, etc... What server would you advise me?
    – So4ne
    Oct 17, 2016 at 8:16
  • Are you using one massive JSON file to serve the data?
    – juniordev
    Oct 17, 2016 at 9:11
  • @juniordev I used to, now I have 11 JSON (depends on the color), biggest size is 18mb (imgur.com/a/RGsan). Gzip is set on my server
    – So4ne
    Oct 17, 2016 at 9:28
  • 1
    Look at NextGIS Web - REST API Lefleat and OpenLayer integration, QGIS renderer, WMS, etc. (docs.nextgis.com/docs_ngweb_dev/doc/toc.html). You can try the cloud version at nextgis.com. If it suits you can install NGW at your own server (docs.nextgis.com/docs_ngweb/source/sysadmin.html) Oct 17, 2016 at 10:03

3 Answers 3

4

Here are 2 suggestions may consider to decrease the loading in UI and request loading:

  • If those points have many attribute values, try to load the Geo field only and not all attributes. (request loading)
  • try to show markers that are contained in map boundary ONLY and hide markers that are out of boundaries by updating the markers visibility when zooming or navigating in map.
1
  • This actually costs me nothing to try for the first point, thank you. For the second point, if I understand well, I have to set all the visibility to none and update with the boundary. I wasn't sure the visibility could change the memory as the markers are supposed to still be there, I'll give it a try and I'll let you know !
    – So4ne
    Oct 17, 2016 at 8:34
2

It is down to the DOM (Document Object Model). If you try to load 150K objects (points) in the browsers' DOM, most browsers will show a "slow running script" dialog and give up depending on the how much memory is at disposal.

I have successfully achieved this by using a service API (you have to design this) call to return all the points in one go to the client in a JavaScript object and then render the points from the object on the client in a batch of say 500 points per iteration (you have to test this on multiple browsers if that is a requirement and adjust the batch size). This will work absolutely 100% and will be seamless to the user. But it is a strange requirement to render so many points at once without clustering as the screen will be too crowded to see the map layer.

Code excerpt (For Bing Maps from about 7 years ago, but the concepts will still apply):

function fGetBingObject(result, userContext) {

    try {
        var markersArray = [];
        if (result.Points) {
            function pMark(result, loopCount) {
                if (loopCount == undefined) {
                    loopCount = 0;
                }

                if (loopCount < result.Points.length) {
                    var batchSize = 500; //100
                    for (currCount = loopCount; currCount < loopCount + batchSize && currCount < result.Points.length; ++currCount) {
                        var myIcon_google;
                        var myPoint = new VELatLong(result.Points[currCount].Latitude, result.Points[currCount].Longitude);
                        var marker = new VEShape(VEShapeType.Pushpin, myPoint);
                        layer =

                            null;
                        if (result.Points[currCount].IconImage != '') {
                            marker.SetCustomIcon(result.Points[currCount].IconImage);

                        }

                        listen(marker, result.Points[currCount].InfoHTML);

                        markersArray.push(marker);

                        if (typeof result.Points[currCount].LayerKey != "undefined") {
                            if (result.Points[currCount].LayerKey != '') {
                                if (result.Points[currCount].LayerKey == 'TEST') {
                                    marker.SetMinZoomLevel(result.Points[currCount].MinZoomLevel);

                                    marker.SetMaxZoomLevel(result.Points[currCount].MaxZoomLevel);

                                    layer = map.GetShapeLayerByIndex(1);

                                    layer.AddShape(marker);

                                }

                            }

                        }

                    }
1
  • That seems pretty workable, I was looking for reduce that DOM. I'll take a look, thanks
    – So4ne
    Oct 18, 2016 at 9:52
0

At smaller scales you should look at reducing the number of vector features being rendered by the map.

You could use pre processed raster tiles from feeds such as mapnik or geoserver WMS. You can also limit the scale at which these features are visible.

As you go into larger scales and want the user interaction you should serve the data via a database, this means the map holds only the information it needs.

options include https://github.com/stefan0722/gs-mvt

or https://github.com/spatialdev/PGRestAPI

or letting mapbox host your data.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.