Converting multilayer KML files to GeoJSON using ogr2ogr

Question

I have a kml file that I'm trying to convert to GeoJSON. However, it turns out there's multiple layers, and ogr2ogr isn't happy.

$ ogr2ogr -f GeoJSON shapes.json shapes.kml
ERROR 6: GeoJSON driver doesn't support creating more than one layer
ERROR 1: Terminating translation prematurely after failed
translation of layer Set 2 (use -skipfailures to skip errors)

Here's a sample of one such multi layer KML file:

<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://www.opengis.net/kml/2.2" xmlns:gx="http://www.google.com/kml/ext/2.2" xmlns:kml="http://www.opengis.net/kml/2.2" xmlns:atom="http://www.w3.org/2005/Atom">
<Document>
    <name>shapes.kml</name>
    <Folder>
        <name>Shapes</name>
        <Folder>
            <name>Set 1</name>
            <Placemark>
                <name>Shape 1-1</name>
                <Polygon>
                    <tessellate>1</tessellate>
                    <outerBoundaryIs>
                        <LinearRing>
                            <coordinates>
                                -81.97209580036224,40.70902818974323,0 -81.97211148995324,40.70705812260233,0 -81.97204657426639,40.70698429917707,0 -81.97170065320428,40.70689762261313,0 -81.97128823249281,40.70678591732805,0 -81.97012759802162,40.70683180630405,0 -81.97011100460675,40.70907869772486,0 -81.97209580036224,40.70902818974323,0 
                            </coordinates>
                        </LinearRing>
                    </outerBoundaryIs>
                </Polygon>
            </Placemark>
            <Placemark>
                <name>Shape 1-2</name>
                <Polygon>
                    <tessellate>1</tessellate>
                    <outerBoundaryIs>
                        <LinearRing>
                            <coordinates>
                                -81.9678256998772,40.70699525577179,0 -81.96710537786173,40.70714109117483,0 -81.9671339172961,40.70964816832036,0 -81.9681094085136,40.70953487937989,0 -81.96849455277558,40.70941143819032,0 -81.9687122721291,40.70922702833104,0 -81.96869523424586,40.70693622825318,0 -81.9678256998772,40.70699525577179,0 
                            </coordinates>
                        </LinearRing>
                    </outerBoundaryIs>
                </Polygon>
            </Placemark>
            <Placemark>
                <name>Shape 1-3</name>
                <Polygon>
                    <tessellate>1</tessellate>
                    <outerBoundaryIs>
                        <LinearRing>
                            <coordinates>
                                -81.97575255412974,40.70901137563123,0 -81.97498009801741,40.70910891904678,0 -81.97366868385656,40.70909911614361,0 -81.97156793031792,40.70919907550063,0 -81.96927940567542,40.70932415940178,0 -81.96979533529213,40.70965324506344,0 -81.97175061906975,40.70968631132654,0 -81.9733120082854,40.70970666237672,0 -81.97576756497214,40.70956450955584,0 -81.97575255412974,40.70901137563123,0 
                            </coordinates>
                        </LinearRing>
                    </outerBoundaryIs>
                </Polygon>
            </Placemark>
            <Placemark>
                <name>Shape 1-4</name>
                <Polygon>
                    <tessellate>1</tessellate>
                    <outerBoundaryIs>
                        <LinearRing>
                            <coordinates>
                                -81.96434152341726,40.70702598013048,0 -81.96356422472108,40.70713349505342,0 -81.96297679852724,40.70696883424459,0 -81.96309807638089,40.70812857198734,0 -81.96343567507532,40.70928783377635,0 -81.96436703230279,40.70914919861734,0 -81.96460689239605,40.70921100214736,0 -81.9650923971045,40.70894923436187,0 -81.96555478608285,40.70876312686659,0 -81.96552441635622,40.70713152110199,0 -81.96434152341726,40.70702598013048,0 
                            </coordinates>
                        </LinearRing>
                    </outerBoundaryIs>
                </Polygon>
            </Placemark>
        </Folder>
        <Folder>
            <name>Set 2</name>
            <Placemark>
                <name>Shape 2-1</name>
                <Polygon>
                    <tessellate>1</tessellate>
                    <outerBoundaryIs>
                        <LinearRing>
                            <coordinates>
                                -81.98834021695214,40.69452103418363,0 -81.98812651732483,40.68978476540717,0 -81.98732201929137,40.68982207193159,0 -81.98748028381483,40.69454809335561,0 -81.98834021695214,40.69452103418363,0 
                            </coordinates>
                        </LinearRing>
                    </outerBoundaryIs>
                </Polygon>
            </Placemark>
            <Placemark>
                <name>Shape 2-2</name>
                <Polygon>
                    <tessellate>1</tessellate>
                    <outerBoundaryIs>
                        <LinearRing>
                            <coordinates>
                                -81.98570709620523,40.69457415615653,0 -81.98548489260064,40.68973423650695,0 -81.98495179849937,40.6899203348901,0 -81.98523118119185,40.6946028701411,0 -81.98570709620523,40.69457415615653,0 
                            </coordinates>
                        </LinearRing>
                    </outerBoundaryIs>
                </Polygon>
            </Placemark>
        </Folder>
    </Folder>
</Document>
</kml>

Are there any suggestions for successfully processing the shapes.kml using existing libraries, or must it be split apart manually? What is your suggested strategy for doing so?

Answer 1

The trick is to get a list of layers in the KML that you can than iterate over individually.

A linux solution (bash) could look like...

for layer in "$(ogrinfo -ro -so -q file.kml | cut -d ' ' -f 2)"
do
    ogr2ogr -f "GeoJSON" "file_${layer}.json" file.kml "${layer}"
done

You might also have an issue of spaces ' ' in the kml layer name.

Those can be substituted (removed or replaced) on the fly with bash:

$ layer1="Site - 123"
$ echo ${layer1// /}
$ Site-123
$ echo ${layer1} | sed 's/ //g'
$ Site-123

Because of this, I highly recommend experimenting with the output of ogrinfo -ro -so -q file.kml | cut -d ' ' -f 2) to make sure it treats the layers in the list of that kml properly. Try to make adjustments to the code so that the spaces don't split up the list too much.

You can see that cut -d ' ' -f 2 will look for spaces (-d ' ') and separate the output into fields where they are found. Then it takes the second field (-f 2) and uses that as the layer name.

Here's an example of the output on a sample sqlite database:

% ogrinfo -ro -so -q ca_bs.sqlite 
1: bs_1250009_0 (Point)
2: bs_2060009_0 (Point)
3: bs_2010009_2 (Multi Polygon)
4: bs_2380009_2 (Multi Polygon)
5: bs_2240009_1 (Multi Line String)
6: bs_2310009_1 (Multi Line String)

% ogrinfo -ro -so -q ca_bs.sqlite | cut -d ' ' -f 2
bs_1250009_0
bs_2060009_0
bs_2010009_2
bs_2380009_2
bs_2240009_1
bs_2310009_1

So this should get pretty close with the sample you provided:

$ ogrinfo -ro -so -q test.kml | cut -d ':' -f2 | sed 's/^ //g'
Shapes
Set 1
Set 2

Answer 2

Mapbox's togeojson converter handles this very nicely. It flattens the contents of all layers/folders into sibling features.

A bonus: unlike the accepted answer using ogr2ogr to extract each layer/folder, togeojson preserves features in the top-level, and not in any directory.

Since it's JavaScript, it can run in your browser at http://mapbox.github.io/togeojson/—fully client-side code, it's not sending your KML to any server to convert. However, it's often important to have a local install for command-line fun.

So to install togeojson, first install node.js (it's small, ~13 MB for the entire runtime, and it's only going to get more useful for GIS). Then use node's package manager npm to install togeojson:

$ npm install -g togeojson

(might need sudo). Finally, convert your file:

$ togeojson test.kml > test.json

The result (as of now, version 0.13.0) has copious whitespace, four spaces per level of indentation. That can be annoying. Here's how to trim it, in the node.js command-line:

$ node
> var fs = require('fs')
> fs.writeFileSync('test-one-space.json', JSON.stringify(JSON.parse(fs.readFileSync('test.json', 'utf8')), null, 1))

This slurps the test.json file, converts it to a JavaScript object ("JSO"?), converts it back to JSON but with one space per indentation level, and writes the result back out to test-one-space.json.

For that unreadable look, eliminate all superfluous whitespace with:

> fs.writeFileSync('test-minified.json', JSON.stringify(JSON.parse(fs.readFileSync('test.json', 'utf8'))))

Answer 3

A further iteration on @SaultDon's answer, using regular expressions for more reliable space handling:

SAVEIFS=$IFS
rm converted/*
KML=/tmp/kml-unzip/doc.kml

layers="$(ogrinfo -q -ro -so "$KML" | perl -pe 's/^[^ ]+ //g and s/ \([^()]+\)$//g')"

# Make sure we don't iterate over spaces in layer names
IFS=$'\n'

for layer in $layers; do
    IFS=$SAVEIFS
    filename=$(basename "$KML")
    filename="${filename%.*}"
    echo "File: $filename, layer $layer"
    ogr2ogr -f "GeoJSON" "converted/${filename}_${layer}.geojson" "$KML" "${layer}"
done
IFS=$SAVEIFS

ogr2ogr doesn't seem to have the ability to merge layers, or even append to an existing GeoJSON. To merge all the layers from the KML into one file, use the geojson-merge utility:

npm install -g geojson-merge

geojson-merge converted/${filename}_*.geojson > "converted/${filename}__merged.geojson"