1

I have 2 geodataframes; one made from polygons (bldg_res_df) and one from centroid points (parcel_res_df). I used .concat to combine them into a single geodataframe to do some calculations.

df_list = [bldg_res_df, parcel_res_df]
combined_df = gpd.GeoDataFrame(pd.concat(df_list, sort=True))

I summarized certain columns based on a shared column (GEOID) between both gdf's.

geoid_sum = combined_df[[ 'GEOID', 'bldg_sqft', 'CensusPop']]
geoid_sum = geoid_sum.groupby('GEOID').agg({'GEOID': 'count', 'bldg_sqft': 'sum', 'CensusPop': 'mean'}).reindex(combined_df['GEOID'])

Then I did my calculations and populated previously empty columns (Pop_By_Area, Tot_Bldg_Sqft, and Census_Bld_Units) with the results. 

combined_df['Pop_By_Area'] = (geoid_sum['CensusPop'].values * 
combined_df['bldg_sqft'])/geoid_sum['bldg_sqft'].values
combined_df['Tot_Bldg_Sqft'] = geoid_sum['bldg_sqft'].values
combined_df['Census_Bld_Units'] = geoid_sum['GEOID'].values

What I want to do now is populate the individual geodataframes with the newly calculated values for the corresponding row. Or, split the combine_df into 2 geodataframes based on geometry type (polygons, points). What is the easiest way to achieve this?

Improve this question
2

You can split this dataframe using either method you described.

To keep your original dataframes you can copy the calculated values by running an apply row-wise and searching the combined dataframe for the same GEOID.

EDIT: This method slows greatly down as the number of items in the dataframes grows since it has to loop through each and every one and search combined_df. This can be mitigated by setting 'GEOID' as the index, as this will allow for a hash scan (like a dictionary or set)

# Set GEOID as the index of combined_df. drop=False, tells the function to keep GEOID in the columns of the dataframe.
combined_df.set_index('GEOID', drop=False, inplace=True)

bldg_res_df['Pop_By_Area'] = bldg_res_df['GEOID'].apply(lambda bldg_geoid: combined_df.loc[bldg_geoid, 'Pop_By_Area'])
parcel_res_df['Pop_By_Area'] = parcel_res_df['GEOID'].apply(lambda parcel_geoid: combined_df.loc[parcel_geoid, 'Pop_By_Area'])

Though a faster, and simpler way of would to be slicing the calculated columns from your combined dataframe, and filtering the geometry types into new dataframes. Geopandas stores geometry types as Shapely objects, so you can make use of the .geom_type attribute of combined_df's geometry column in a .loc call. 

points_df = combined_df.loc[combined_df['geometry'].geom_type == 'Polygon', ['GEOID', 'HU_Pop', 'PARCEL_ID', 'Pop_By_Area', 'STORY_NBR', 'Tot_Bldg_Sqft', 'bldg_sqft', 'geometry']]]
polygon_df = combined_df.loc[combined_df['geometry'].geom_type == 'Point', ['GEOID', 'HU_Pop', 'PARCEL_ID', 'Pop_By_Area', 'STORY_NBR', 'Tot_Bldg_Sqft', 'bldg_sqft', 'geometry']]]
Improve this answer
5
  • I am trying the first option, but its taking a while. I am a little unsure about your second method. Let's say I want both points_df and polygon_df to have the same columns(['GEOID', 'HU_Pop', 'PARCEL_ID', 'Pop_By_Area', 'STORY_NBR', 'Tot_Bldg_Sqft', 'bldg_sqft', 'geometry']). The only difference is that one will have geometry type=points and one will have geometry type=polygons. Can you clarify the specific syntax? – gwydion93 Jul 22 '19 at 17:41
  • 1
    Updated my response to provide more clarity into the slicing option. Also sped up the lookup of the apply option by indexing combined_df on GEOID – Nate Jul 22 '19 at 18:14
  • This almost works. Since I kept it as a geodataframe, it only has one geometry column though. – gwydion93 Jul 22 '19 at 18:27
  • 1
    My mistake, I had misread earlier. You can use .geom_type attribute of the geometry column to filter out point and polygon geometries – Nate Jul 22 '19 at 18:47
  • OK, this worked great! I had tried using the .geom_type earlier but I was applying it incorrectly. This is quicker than the first method (which also works) and runs over a mil + dataset in 10-15 minutes. – gwydion93 Jul 23 '19 at 1:36
1

You can use merge to populate your original geodataframes, which I assume will be significantly faster than looping over proposed by @Nate.

bldg_res_df = bldg_res_df.merge(combined_df[['GEOID', 'Pop_By_Area', 'Tot_Bldg_Sqft', 'Census_Bld_Units']], on='GEOID', how='left')
parcel_res_df = parcel_res_df.merge(combined_df[['GEOID', 'Pop_By_Area', 'Tot_Bldg_Sqft', 'Census_Bld_Units']], on='GEOID', how='left')
Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.