1

I am working with the National Wetlands Inventory, a set of shapefiles that describe all wetlands in the United States. There are 50 states, and each state has at least one shapefile (some have more than one). I want to make a national map from all the data in these shapefiles.

My current approach is to load each file in a loop and plot it to a set of shared axes, so at the end I have a national map. Here is a pseudocode example of how this works:

import matplotlib.pyplot as plt
import geopandas as gpd 

fig, ax = plt.subplots()

for shp in shpfile_paths:
    this_shp = gpd.read_file(shp)
    this_shp.plot(ax=ax)

plt.show()

This works ok, but it is very slow, and if the loop errors, I have to start all over again. I have tried saving the plot as a pickle using pickle.dump() after each iteration of the loop, so in case of an error I can pick up where I left off, but this creates a huge pickle file which eventually started causing IO errors.

Any suggestions for how I might do this more efficiently?

1
  • Merge all the files, then plot the merge output
    – Bera
    Commented Sep 15, 2023 at 4:58

1 Answer 1

1

I am not sure if this is what you had in mind, but it seems much more effective to first merge the shapefiles (I assume they have a similar structure) and then plot one large geodataframe:

shp_gdfs = []
for shp_path in shpfile_paths:
    shp_gdf = gpd.read_file(shp_path)
    shp_gdfs.append(shp_gdf)

total_gdf = gpd.GeoDataFrame(pd.concat(shp_gdfs, ignore_index=True), crs=shp_gdfs[0].crs)
total_gdf.plot()

(thanks https://gis.stackexchange.com/a/162661/23224 for the concatenation code)

I do doubt if one very large map for the whole of the United States will ever fit nicely in a matplotlib plot. I recommend you store the total_gdf as a shapefile or similar and open it in e.g. QGIS to make a nice map.

1
  • Thanks @JohanB, I can try this. It seems to me like holding all the data in memory by concatenating the shapefiles is not solving the core problem, but it's possible there is no solution that will allow me to do this in the way I was hoping.
    – sdg
    Commented Nov 1, 2022 at 22:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.