Split planet.osm into multiple files based on UTM grid

Question

Starting with the full OSM Planet data, I'm trying to create smaller, individual osm/pbf files, or eventually Spatia/SQLite or GeoJSON files, that are split on each UTM grid zone. I'm using osmfilter to trim down the data, and I imagine with a UTM zone shapefile, you could create a filter for each UTM zone in a batch process, perhaps using osmosis or mapsplit (but I don't know how, nor do I know where to get the shapefiles or which format to use with osmfilter).

The reason is that I want my individual osm/pbf files to be somewhat evenly sized, and not proportional to the size of the country (for example, the US might be 10 GB and Luxembourg 10 KB).

Is there anyway to do this using the OSM tools like osmfilter?

Or could I do this after I've converted to Spatia/SQLite with ogr2ogr?

update UTM zones shapefile is available here, although I still don't know how to create individual files from planet.osm based on this shapefile

Here's what Europe looks like in UTM - so each of these zones would be one file, derived from planet.osm with osmfilter applied to keep only certain tags.

Answer 1

I use Python predominately, so here's a skeleton workflow I'd use for this. There are probably better ways to go about it, but this one will get you what you want.

1) Load your UTM shapefile using shapely or geopandas (which uses shapely).

2) I haven't looked at the UTM shapefile, but I assume each cell is defined by four corner lon,lat points. If so, move to next step. If not, you can use the shapely geometry method .box to return the bounding box.

3) Use Osmosis to do the heavy lifting, see here for examples on extracting by area/bounding box. I would loop through each polygon, grab the corner coordinates, the make a subprocess call to osmosis. The relevant options for osmosis are:

--bounding-box top=49.5138 left=10.9351 bottom=49.3866 right=11.201

You can loop through each feature of the shapefile, grab coordinates, create subprocess string, call subprocess. It's easy parallelizable as this is probably a bigger job.

I don't know if there's any value in buffering your bounding boxes a little or if that would cause issues with whatever application you're building. shapely (and geopandas) has easy functions for that. There could be rounding errors or something strange which leave tiny gaps between cells; the safest thing to do is run a couple before cranking the whole planet file. Also, when I did a global extraction on the planet file, I found it faster to unzip the planet file first rather than do it on the fly. But that takes ~1TB of space.

Answer 2

Using Mapsplit is efficient for this. You should convert the UTM zones into poly files then you can batch process it in a loop in command line using each file as the --polygon=file parameter in Mapsplit.

ogr2poly.py can help to convert shapefiles to poly files. More options are available at Converting_to.2Ffrom_POLY_format