3

I would like to query a large bounding box in Openstreetmap using the overpass API from within python. A solution similar to this one would be ideal.

My extent, however is very large, and I would like to determine if there is any native "overpass API" method to loop through "sub-bounding boxes" and extract data for each smaller bounding box.

I can think of ways to do this in python but I would rather take care of this at the Overpass API level. Surely they have devised a way to avoid errors like these:

runtime error: Query ran out of memory in "query" at line 1. It would need at least 531 MB of RAM to continue.

Are there any Overpass-specific capabilities which will successfully download data for large extents? Which commands/queries are of use here?

Improve this question
3
  • 1
    Do you want to dowload specific data (for example only peaks or only shop=seafood) or all data from large area? What you consider "large extent" - entire world? Single city? Europe? Switzerland?
    – reducing activity
    Oct 26, 2018 at 9:48
  • I want to download specific data, namely the highway bridges. "Large extent" in this case is referring to the continent of Europe. I have a bounding box defined 35.359872101102724, -29.628211426818392, 70.90180289857001, 47.014599265244506
    – user32882
    Oct 26, 2018 at 10:22
  • 1
    Better download a Europe PBF extract, and use some tool like osmium to extract those bridges. This may be faster after all.
    – mmd
    Oct 26, 2018 at 15:47

1 Answer 1

Reset to default
2

if there is any native "overpass API" method to loop through "sub-bounding boxes" and extract data for each smaller bounding box.

No.

for large extents

See https://wiki.openstreetmap.org/wiki/Overpass_API#Resource_management_options_.28osm-script.29 for how to request more resources. It will allow you to run queries for larger areas but server will refuse too greedy queries.

You may need to either split query into multiple or download data extract and process it locally, maybe on your Overpass instance.

/*
This has been generated by the overpass-turbo wizard.
The original search was:
“bridge=yes and highway=*”
*/
[out:json][timeout:250][maxsize:4073741824];
// gather results
(
  // query part for: “bridge=yes and highway=*”
  // note - real query would need to accept more values than just "yes"
  node["bridge"="yes"]["highway"]({{bbox}});
  way["bridge"="yes"]["highway"]({{bbox}});
  relation["bridge"="yes"]["highway"]({{bbox}});
);
// print results
out body;
>;
out skel qt;

run on the entire Europe in https://overpass-turbo.eu/s/D8t still fails - but with a different message.

Improve this answer
3
  • The error message reads "It would need at least 3892 MB of RAM to continue." - so what you're proposing here doesn't really help - it still consumes too much memory.
    – mmd
    Oct 26, 2018 at 16:09
  • 1
    @mmd I know, but (1) it helps with smaller querries - requiring between 531 MB and 3892 MB, maybe someone will google it and find an useful answer to very similar problem (2) is given as proof that this query is not able to run in one part, even after requesting as much as possible from Overpass Turbo server and to make clear that Overpass is likely a bad tool for this task.
    – reducing activity
    Oct 26, 2018 at 19:41
  • 1
    Ok, the memory consumption issue may change in a future Overpass API release. A dev version already shows quite some lower memory consumption.
    – mmd
    Oct 27, 2018 at 9:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.