1

Is anyone clever enough to extract longs and lats for the points shown at:

https://www.northernpowergrid.com/generation-availability-map

What would be even better is finding out if those points hold the attributes which can be viewed if you download the excel heat map data from the same webpage. I've called Northern Power Grid and they don't have location data for this dataset which must be complete rubbish.

I'm not asking you to do it for me..(although that would be extremely kind) but hopefully you could give me a nudge in the right direction.

3

All credit to @marc-pfister who initially pointed to the POST query. I just took it a step further.

Here's a screencast showing what I did: http://screencast-o-matic.com/watch/cbjor2ldMx

Basically, take the JSON data from the raw response, view it in a readable format (optional step), then convert it to .csv for consumption (or use it in JSON format).

The lat/long is certainly there, it's just obscured from the excel file that they provide.

I'm not affiliated with any of the services that I used to find the solution. I work with addresses every day and just like solving problems.

  • Jeffrey....This is incredibly kind of you to go to this extent, it's by far the most effective response I've ever received on this forum, thank you so much. Is there any way to break the JSON down into bite size chunks before converting into CSV? (I have no experience in coding JSON) – James B Aug 16 '17 at 9:32
  • There's not really a small-chunk way to consume the JSON data. lt's in an array, or group of objects and each object has elements. So, the first object has a list of the column that you can expect in the following objects and then you have data within the object elements. You can inspect it a little more by pasting that JSON data in here: jsonviewer.stack.hu – Jeffrey Aug 16 '17 at 15:56
  • I managed to convert the entire line of JSON (1.06Mb) for free using onlinejsonconvert.com/json-csv.php. I'm trying to do the same for holz-kraft.com/en/references/reference-map.html but struggling! The most likely file I found to hold the lat and long data for these pins is the file "ViewportInfoService". Although the response is not in JSON format so perhaps it's not the right file....How do you go about finding the correct file so quickly like you did with Northern Power Grid? Obviously experience has to help. – James B Aug 18 '17 at 8:43
  • On the holz-kraft.com page, you will find the input variables in a different way. Inspect the reference-map.html file and you'll see all of the lat/lon values that are in use on the map within the html file itself. Like this: var phocaLatLng = new google.maps.LatLng(49.89463439573423, 9.11865234375); Different way to submit the values to GoogleMaps – Jeffrey Aug 21 '17 at 18:38
3

If you open up the developer tools in your browser and watch the network tab while the map loads, there's a POST query to /primarysubstationquery that returns a JSON object that might have some useful data in it.

If that's what you need, then unless you're ok with fetching it manually you will have to look into web scraping or browser automation tools to retrieve it, since that query probably won't return unless you've gone through the motions of accepting the disclaimer.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.