Writing GeoDataFrame to GeoJSON without Null values in GeoPandas

Question

I have a GeoPandas GeoDataFrame:

                  geometry col0 col1 col2  col3
0  POINT (0.00000 3.00000)    A    A    A   NaN
1  POINT (1.00000 4.00000)    B    B  NaN   NaN
2  POINT (2.00000 5.00000)    C  NaN  NaN   NaN

Reproducible via:

import geopandas as gpd
import numpy as np

df = \
gpd.GeoDataFrame.from_dict(
    dict(
        geometry=gpd.points_from_xy([0,1,2],[3,4,5]),
        col0=["A","B","C"],
        col1=["A","B",np.nan],
        col2=["A",np.nan,np.nan],
        col3=[np.nan,np.nan,np.nan],
    )
).set_crs(4326)

I would like to create a GeoJSON file from it, without having the key-value pairs where the value is null. The desired GeoJSON file:

{
"type": "FeatureCollection",
"crs": { "type": "name", "properties": { "name": "urn:ogc:def:crs:OGC:1.3:CRS84" } },
"features": [
{ "type": "Feature", "properties": { "col0": "A", "col1": "A", "col2": "A" }, "geometry": { "type": "Point", "coordinates": [ 0.0, 3.0 ] } },
{ "type": "Feature", "properties": { "col0": "B", "col1": "B" }, "geometry": { "type": "Point", "coordinates": [ 1.0, 4.0 ] } },
{ "type": "Feature", "properties": { "col0": "C" }, "geometry": { "type": "Point", "coordinates": [ 2.0, 5.0 ] } }
]
}

I try to create this file via df.to_file("test.geojson"). The created test.geojson:

{
"type": "FeatureCollection",
"crs": { "type": "name", "properties": { "name": "urn:ogc:def:crs:OGC:1.3:CRS84" } },
"features": [
{ "type": "Feature", "properties": { "col0": "A", "col1": "A", "col2": "A", "col3": null }, "geometry": { "type": "Point", "coordinates": [ 0.0, 3.0 ] } },
{ "type": "Feature", "properties": { "col0": "B", "col1": "B", "col2": null, "col3": null }, "geometry": { "type": "Point", "coordinates": [ 1.0, 4.0 ] } },
{ "type": "Feature", "properties": { "col0": "C", "col1": null, "col2": null, "col3": null }, "geometry": { "type": "Point", "coordinates": [ 2.0, 5.0 ] } }
]
}

This is not what I want, since the properties array contains key-value pairs where the value is null:

"properties": { "col0": "A", "col1": "A", "col2": "A", "col3": null }
"properties": { "col0": "B", "col1": "B", "col2": null, "col3": null }
"properties": { "col0": "C", "col1": null, "col2": null, "col3": null }

I could create the GeoJSON line by line, but that seems tedious. I could theoretically also create a GeoJSON with the nulls, and then produce another GeoJSON without the nulls. In my real-world use case, I have a lot of different columns, almost all of them have rows almost exclusively set to null, so this is not very practical. I am also open to using another file format as GeoPandas' output, followed by a conversion to GeoJSON by ogr2ogr for example. The naive

df.to_file("test.gpkg")

followed by

ogr2ogr test.geojson test.gpkg

did not solve the issue (which is expected, I need to write a more specific ogr2ogr command to have the desired output, but I haven't yet figured out what this command is exactly).

This thread is on a similar topic, asking how to remove nulls from an existing GeoJSON. I would like to create one without the nulls being there in the first place from the GeoDataFrame.

What is an easy way to achieve the above-described result?

Answer 1

with open("test.geojson","w") as f:
    f.write(df.to_json(na="drop"))

results in this test.geojson:

{"type": "FeatureCollection", "features": [{"id": "0", "type": "Feature", "properties": {"col0": "A", "col1": "A", "col2": "A"}, "geometry": {"type": "Point", "coordinates": [0.0, 3.0]}}, {"id": "1", "type": "Feature", "properties": {"col0": "B", "col1": "B"}, "geometry": {"type": "Point", "coordinates": [1.0, 4.0]}}, {"id": "2", "type": "Feature", "properties": {"col0": "C"}, "geometry": {"type": "Point", "coordinates": [2.0, 5.0]}}]}

which is similar enough to the desired solution so that it solves my problem.

Answer 2

If we are ok with creating a GeoJSON with nulls, and then creating another one, then this also works:

jq '.features |= map(.properties |= with_entries(select(.value != null)))' w_nulls.geojson > wo_nulls.geojson