0

I have the website about the planet occultations by Moon.

https://eco.mtk.nao.ac.jp/cgi-bin/koyomi/occulx_p_en.cgi

There are "Occultation maps", which are very nice: enter image description here

I tried to use the Web Scraper for Chrome, but it looks like it grabs the stuff to Excel only.

I would like to extract this occultation map to a KML file or similar. Is it possible?

4
  • 1
    In short, you need the red lines as a spatial file, right? – Kadir Şahbaz May 10 at 22:58
  • Yes indeed. Only the red lines. Next, they could be filled up in QGIS. – MKR May 11 at 8:26
  • Hello @MKR I have proposed an answer if anything is not clear don't hesitate to tag me, I could then adapt my answer. – Corentin Lemaitre May 11 at 14:52
  • Hi Sir. I know, let me give it a go tomorrow. I was busy today. It looks like we will be on the right track. Thank you very much. – MKR May 11 at 15:13
6
+150

The lines data displayed in the map is in a .js file named gmaps202101.js. You can get a GeoJSON text of all polylines coded in the file.

  • Open Dev Tools Ctrl+Shift+C > Sources Tab > Page Tab.

  • Navigate to eco.mtk.nao.ac.jp > koyomi > koyomix/poclgmap/koyomix > gmaps202101.js file.

  • Copy all content of the file.

  • Go to Snippets tab > New snippet. Paste the code.

  • Add the following lines to set_Map function (right above return)

    lines = [line0, line1, line2, line3, line4, line5, line6];    
    exportGeoJson(lines);
    
  • Add the following lines to the end of the file and run (Ctrl+Enter). This prints to the console a JSON object.

    function exportGeoJson(lines) {
        var geoJson = {
            "type": "FeatureCollection",
            "features": []
        };
    
        for (var i = 0; i < lines.length; i++) {
    
            var lineFeature = {
                "type": "Feature",
                "geometry": {
                    "type": "LineString",
                    "coordinates": []
                },
                "properties": {}
            };
    
            path = lines[i].getPath();
            for (var j = 0; j < path.getLength(); j++) {
                var pt = path.getAt(j);
                lineFeature.geometry.coordinates.push([
                    pt.lng(), pt.lat()
                ]);
            }
    
            geoJson.features.push(lineFeature);
        }
    
        console.log(geoJson);
    }
    
    set_Map();
    
  • Copy/paste it into a text file. Save as .geojson.

Demo:

enter image description here

9
  • A better solution would be writing a PyQGIS script like J.Monticolo's answer to your previous question. – Kadir Şahbaz May 12 at 10:13
  • My console is empty, here is the problem, there is no script snippet visible as you shown me in the demo version – MKR May 14 at 9:57
  • @MKR What do you mean by "no script snippet visible"? You should copy all gmaps202101.js's content to a "New snippet", then copy/paste the scripts above to the "New snippet". Help for snippet – Kadir Şahbaz May 14 at 10:07
  • Yes I did exactly as you shown me. imgur.com/a/4f49kS1. I have nothing to click in the console. – MKR May 14 at 10:12
  • @MKR Are you sure you added the last line -> set_Map(); – Kadir Şahbaz May 14 at 10:14
5

Here is a solution for your question (but may be the best is ask for the data to the producer).

First I searched for the data that is shown in red on the map. If you use chrome you can access dev tools then in the source tab you will find all the files used in the website. Go to folder eco.mtk.nao.ac.jp then folder koyomi then koyomixpoclgmap/koyomix then the js file (the file name depend on the value you have used as parameter).

enter image description here

This file is the javascript command used to produce the map. In the function called set_Map there is variable named points0 that contains the points to produce one of the line in red. Points location are stored as encoded format (see documentation here). The variable line0 is used to create a google maps polyline to be traced on the map. This process is repeated for every line in the map (points1 line1 points2 line2...).

The next step is to extract from the JS file the encoded value. You can use notepad find and replace to delete the part of the file you don't need and keep only the encoded values.

Then you have to decode the value to produce a GIS file. You can choose different solution to do it, the choice will depend on the process you are familiar with. Here are two different options you can use :

  1. Post Gis function to create line from encoded polyline. To use it you need to have access to a postgis database or to install postgis database in your computer. Use the encoded values to build a sql querry, here is the documentation.
  2. javascript library made by mapbox to transform encoded value to geojson file. It could work in the console or in the browser here is the documentation

After this step you get your geodata and you can use it in QGIS.

1
  • Hi, Your answer is nice but, seems to be too difficult to me. First of all: 1. How to decode the coordinates in Javascript? 2. How to create the line in PostGIS? I am a newbie so far 3. Same to JavaScript Mapbox. – MKR May 14 at 13:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.