Key error when trying to loop through geodataframe in python

Question

I have a geodataframe 55 polygons for this example. Within this geodataframe is a column called “Colors” that simply has the string values either “Red”, “Blue”, or “Green”, and “Values” column that has float values. What I want to do is loop through the rows in this geodataframe, where each row represents a unique, separate polygon, and then drop all rows that have the same color as the target polygon, and then create a kernel density estimation raster from those remaining polygons, which includes the target polygon. I want each produced kde raster to be “weighted” by the “Values” column. And so I am aiming to produce 55 separate rasters, with one produced for each iteration of the loop. To create these kde rasters I am using the spatial_kernedel_density() function from the spatial-kde package: https://pypi.org/project/spatial-kde/

Here is what I am trying:

#for each polygon drop rows from polygons with same color

index = 0

for pos, row in polygons.iterrows():
    
    index += 1
    index_str = str(index)
    
    color_drop = str(row.Color)
    polygons_filtered = polygons[~polygons.Color.str.contains(color_drop)]
    
    polygons_filtered.boundary.plot()

    #And then create a separate kde raster for each unique filtered geodataframe
    spatial_kernel_density(
        points=polygons_filtered,
        radius=402.3,
        output_path="C:/Users/MyName/Documents/result_kde_" + index_str + ".tif",
        output_pixel_size=10.0,
        output_driver="GTiff",
        weight_col="Value",
        scaled=False,
    )

When I run this though, I receive this error message: KeyError: 19

I cannot figure out where this is coming from. I thought maybe somehow the 19th polygon in my geodataframe is corrupted, but just dropping the 19th row in my geodataframe still produced the same error.

This confuses me, because I am able to succesffully create my kde raster from just the original geodataframe, without filtering by color, like so:

spatial_kernel_density(
     points=polygons,
     radius=402.3,
     output_path="C:/Users/MyName/Documents/result_kde.tif",
     output_pixel_size=10.0,
     output_driver="GTiff",
     weight_col="Value",
     scaled=False,
)

How might I go about fixing this key error so I can successfully produce all 55 of my kde rasters?

Full traceback error:

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
File ~\Documents\MyName\lib\site-packages\pandas\core\indexes\base.py:3621, in Index.get_loc(self, key, method, tolerance)
   3620 try:
-> 3621     return self._engine.get_loc(casted_key)
   3622 except KeyError as err:

File ~\Documents\MyName\lib\site-packages\pandas\_libs\index.pyx:136, in pandas._libs.index.IndexEngine.get_loc()

File ~\Documents\MyName\lib\site-packages\pandas\_libs\index.pyx:163, in pandas._libs.index.IndexEngine.get_loc()

File pandas\_libs\hashtable_class_helper.pxi:2131, in pandas._libs.hashtable.Int64HashTable.get_item()

File pandas\_libs\hashtable_class_helper.pxi:2140, in pandas._libs.hashtable.Int64HashTable.get_item()

KeyError: 12

The above exception was the direct cause of the following exception:

KeyError                                  Traceback (most recent call last)
Input In [7], in <cell line: 5>()
     15     stores_zip_filtered.plot()
     17 #     try:
---> 18     spatial_kernel_density(
     19         points=polygons_filtered,
     20         radius=402.3,
     21         output_path="C:/Users/MyName/Documents/result_kde_" + index_str + ".tif",
     22         output_pixel_size=10.0,
     23         output_driver="GTiff",
     24         weight_col="Value",
     25         scaled=False,
     26     )

File ~\Documents\MyName\lib\site-packages\spatial_kde\kde.py:103, in spatial_kernel_density(points, radius, output_path, output_pixel_size, output_driver, weight_col, scaled)
    101 for row in kde_pnts.itertuples():
    102     centre = Point(xy[row.Index])
--> 103     distances = [centre.distance(points.at[i, "geometry"]) for i in row.nn]
    105     weights = None
    106     if weight_col:

File ~\Documents\MyName\lib\site-packages\spatial_kde\kde.py:103, in <listcomp>(.0)
    101 for row in kde_pnts.itertuples():
    102     centre = Point(xy[row.Index])
--> 103     distances = [centre.distance(points.at[i, "geometry"]) for i in row.nn]
    105     weights = None
    106     if weight_col:

File ~\Documents\MyName\lib\site-packages\pandas\core\indexing.py:2270, in _AtIndexer.__getitem__(self, key)
   2267         raise ValueError("Invalid call for scalar access (getting)!")
   2268     return self.obj.loc[key]
-> 2270 return super().__getitem__(key)

File ~\Documents\MyName\lib\site-packages\pandas\core\indexing.py:2221, in _ScalarAccessIndexer.__getitem__(self, key)
   2218         raise ValueError("Invalid call for scalar access (getting)!")
   2220 key = self._convert_key(key)
-> 2221 return self.obj._get_value(*key, takeable=self._takeable)

File ~\Documents\MyName\lib\site-packages\pandas\core\frame.py:3622, in DataFrame._get_value(self, index, col, takeable)
   3616 engine = self.index._engine
   3618 if not isinstance(self.index, MultiIndex):
   3619     # CategoricalIndex: Trying to use the engine fastpath may give incorrect
   3620     #  results if our categories are integers that dont match our codes
   3621     # IntervalIndex: IntervalTree has no get_loc
-> 3622     row = self.index.get_loc(index)
   3623     return series._values[row]
   3625 # For MultiIndex going through engine effectively restricts us to
   3626 #  same-length tuples; see test_get_set_value_no_partial_indexing

File ~\Documents\MyName\lib\site-packages\pandas\core\indexes\base.py:3623, in Index.get_loc(self, key, method, tolerance)
   3621     return self._engine.get_loc(casted_key)
   3622 except KeyError as err:
-> 3623     raise KeyError(key) from err
   3624 except TypeError:
   3625     # If we have a listlike key, _check_indexing_error will raise
   3626     #  InvalidIndexError. Otherwise we fall through and re-raise
   3627     #  the TypeError.
   3628     self._check_indexing_error(key)

KeyError: 12

Answer 1

Looks like an indexing issue with spatial_kde package. Try using the reset_index method on the polygons like so:

spatial_kernel_density(
     points=polygons_filtered.reset_index(),
     radius=402.3,
     output_path="C:/Users/MyName/Documents/result_kde.tif",
     output_pixel_size=10.0,
     output_driver="GTiff",
     weight_col="Value",
     scaled=False,
)

For why this is so, checkout at this answer about pandas loc vs iloc. https://stackoverflow.com/a/31593712