0

I've been asked to produce a geopackage file in a specified format where one attribute is specified to be int64 nullable, and my case includes NULL values. Trying in GeoPandas, my problem is that I don't manage to create a geodataframe with a dtype=int64 containing NULL values.

I've tried adding a list of integer and None values, which becomes floats in the dataframe:

>>> import geopandas as gp
>>> from shapely.geometry import Point
>>> outdf = gp.GeoDataFrame(geometry=[Point(1, 2), Point(2, 1)])
>>> outdf['nulled_int'] = [0, None]
>>> outdf['nulled_int']
0    0.0
1    NaN
Name: nulled_int, dtype: float64

Trying to cast it to integer gives an error:

>>> outdf['nulled_int'] = outdf['nulled_int'].astype(int)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/a002001/develop/lib64/python3.6/site-packages/pandas/core/generic.py", line 5698, in astype
    new_data = self._data.astype(dtype=dtype, copy=copy, errors=errors)
  File "/home/a002001/develop/lib64/python3.6/site-packages/pandas/core/internals/managers.py", line 582, in astype
    return self.apply("astype", dtype=dtype, copy=copy, errors=errors)
  File "/home/a002001/develop/lib64/python3.6/site-packages/pandas/core/internals/managers.py", line 442, in apply
    applied = getattr(b, f)(**kwargs)
  File "/home/a002001/develop/lib64/python3.6/site-packages/pandas/core/internals/blocks.py", line 625, in astype
    values = astype_nansafe(vals1d, dtype, copy=True)
  File "/home/a002001/develop/lib64/python3.6/site-packages/pandas/core/dtypes/cast.py", line 868, in astype_nansafe
    raise ValueError("Cannot convert non-finite values (NA or inf) to integer")
ValueError: Cannot convert non-finite values (NA or inf) to integer

Starting off from a Pandas series of integer type also get casted to float automatically:

>>> import pandas as pd
>>> outdf['nulled_int'] = pd.Series(dtype=int)
>>> outdf['nulled_int']
0   NaN
1   NaN
Name: nulled_int, dtype: float64

The geopackage format seems happy with nullable integer fields, I've made myself a small testfile with two integer attribute fields: nulled_int containing a null value and integers containing only 0 and 1:

$ ogrinfo riket_test.gpkg -al -geom=no -noextent -nomd
    
INFO: Open of `riket_test.gpkg'
      using driver `GPKG' successful.

[...]

nulled_int: Integer64 (0.0)
integers: Integer64 (0.0)
OGRFeature(riket_test):1
  nulled_int (Integer64) = 1
  integers (Integer64) = 1

OGRFeature(riket_test):2
  nulled_int (Integer64) = (null)
  integers (Integer64) = 0

Reading this file into GeoPandas ends up in the same result as in the attempts to create it above: the field containing the NULL value gets casted to float:

>>> df = gp.read_file('riket_test.gpkg')
>>> df['nulled_int']
0    1.0
1    NaN
Name: nulled_int, dtype: float64
>>> df['integers']
0    1
1    0
Name: integers, dtype: int64
>>> 
7
  • Did you try None? Aug 17, 2022 at 14:44
  • No I had not but unfortunatly the result is the same.
    – jarveliu
    Aug 17, 2022 at 14:50
  • I think that NaN and null do not mean the same in SQLite. See for example system.data.sqlite.org/index.html/tktview/…. Or maybe it is, see stackoverflow.com/questions/20619957/…. Try to update your NaN values into null with SQL and see what happens.
    – user30184
    Aug 17, 2022 at 14:51
  • I agree, they have not the same meaning. What I should store is null rather than NaN wich is what I kind of expected while using the empty pandas series. Anyway trying to add a None value (which I beleave is the python equivalent to null) ends up as NaN in the dataframe.
    – jarveliu
    Aug 17, 2022 at 15:18
  • Changing format in SQL in the generated file afterwards is ofcourse a workaround but my thought with the question was to see if it is possible within the geopandas environment.
    – jarveliu
    Aug 17, 2022 at 15:21

1 Answer 1

0

You could try converting your column to nullable integer data type

outdf['nulled_int'] = outdf['nulled_int'].astype('Int64')

It will get You a bit further:

>> outdf
                  geometry  nulled_int
0  POINT (1.00000 2.00000)           0
1  POINT (2.00000 1.00000)        <NA>

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.