1

I am trying to use http://overpass-turbo.eu ... Something as

[out:csv(::id, wikidata)];
way["wikidata"];
out center;

PS: the answer can restrict area to a country by a kind of area["ISO3166-1"="BR"][boundary=administrative] filter... Or more specific administrative area.

A table with the pair of osm_id and wikidata_id of each tagged way of the map... And I need also to limit (use a SQL-like LIMIT clause) to test some other variants before request full query.


NOTE 1 - Other solutions

Perhaps taginfo.openstreetmap.org also can generate this CSV, but I not see how to.

NOTE 2 - Overpass answer at comments

I am testing regegular expression but was ignored

[out:csv(::id, wikidata)][timeout:25];
(
  way["wikidata"~"Q[0-9]+"]({{bbox}});
);
out body;
>;
out skel qt;

The answer was posted by @mmd as comment http://overpass-turbo.eu/s/A84

[out:csv(::id, wikidata)][timeout:25];
way["wikidata"~"Q[0-9]+"]({{bbox}});
out 10;

where out 10 is the answer for the "LIMIT clause" problem for tests or sampling...

NOTE 3 - Other solution for non-limited CSV

Overpass will not run without a "limit parameter", so @mmd suggested other solution, that is not Overpass... I agree.

PS: the answer can also use terminal and some tool as the suggested by @mmd, as osmium-tags-filter is perhaps a better solution.

  • 1
    266 976 rows is really too much for an online query. Either restrict it by a bounding box, or better use a Planet dump to extract the data. Your second query needs different out statements, but basically works: overpass-turbo.eu/s/A83 – mmd Jul 7 '18 at 11:16
  • Hi @mmd it seems that the "limit clause" is timeout, so I edit and using it at an example, that will generate a good sample... But now the problem is to filter. – Peter Krauss Jul 7 '18 at 11:18
  • You can restrict the number by using out 10; as an example. timeout is no limit clause, it will return either all or nothing. – mmd Jul 7 '18 at 11:20
  • 1
    Again, there's no meaningful way to download 266976 entries for the whole planet this way. Please use some other approach. – mmd Jul 7 '18 at 11:32
  • 1
    So what exactly are you trying to do in the first place? If you want to run this whole thing for the planet, even split into multiple queries, better look at downloading a full planet PBF file and process it via osmium-tool: github.com/osmcode/osmium-tool/blob/master/man/… – mmd Jul 7 '18 at 11:44
1

There are a lot of discussion and no answer because Overpass is only for a first prospection... The "full data into a CSV" solution need a specialized tool and/or to extract directelly from "planet" XML file. So, this answer is kind of "Other solutions".

Transforming the map's XML-dump into desired CSV

At the time when I posted the question, I posted also a Github's draft solution, as simple-osmWd2csv v0.0.1. Summary:

  1. Get the full XML dump of region, that in general is a big .osm.pbf file, so you need also to extract with osmium cat -f osm file.osm.pbf -o file.osm

  2. Pre-parse: simplify the big XML to reduce the CPU time in the next step.

  3. Parse the residual XML generationg a CSV as result.

Ideal will be to develop an osmium module, parsing directally the PBF file... For more details and other solutions see (and help to enhance!) https://github.com/OSMBrasil/simple-osmWd2csv

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.