Convert JSON to GeoDataFrame and append both geometry and filename

Question

I am currently running an experiment on automating a task by merging information from various file types and doing a number of operations to achieve this.

Currently, I am reading a list of GeoJSONs and trying to build a GeoDataFrame that contains both the geometry and the filename.

Example of file name structure in folder is: event1.json

The desired geodataframe should look like this:

    index     geometry                      event_name
    0         LINESTRING1(coordinates...)   event1
    1         LINESTRING2(coordinates...)   event2
    2         LINESTRING3(coordinates...)   event3

I am able to read the JSONs and create the geometries, but I am not able to assign the matching filename.

     import os, geopandas as gpd

     event_tracks = gpd.GeoDataFrame(columns=['geometry', 'eventname'])

     directory = os.getcwd()

     
     for dir_root, dir_dir, js_file  in os.walk(directory):
       for f in js_file:
         if f.endswith(".json"):
         fname = os.path.basename(f)
         fname, fext = fname.split('.')
         print (fname)
         print ('Reading Track Path JSON ' + f)
         with open(f, 'r') as path_f:
           event_tracks = gpd.read_file(path_f).append(event_tracks)

The code above works for geometries and generates a geodataframe with valid geometries. However, I cannot assign the event name into the column. So far, I've tried to:

• Create list of names and append list elements
• Reindex and concatenate two different dataframes

However, it either returns a NaN-filled column or an error such as: TypeError: cannot concatenate object of type '<class 'list'>'; only Series and DataFrame objs are valid.

Is there any workaround to do this?

Answer 1

Assuming that the event name is equals the file name, you would need to assign the event name to each newly "imported" dataframe before appending.

import os
from pathlib import Path
import geopandas as gpd

event_tracks = gpd.GeoDataFrame(
    columns=["geometry", "eventname"],
    geometry="geometry",
    crs="EPSG:4326"  # replace with correct CRS
)

search_path = os.getcwd()

for path in Path(search_path).glob("**/*.json"):
    
    print(f"Reading Track Path JSON {path.name}")

    tracks = gpd.read_file(path.absolute())
    tracks["eventname"] = path.stem  # file name
    
    event_tracks = tracks.append(event_tracks)

Some other notes:

• There is no need to open the file since gpd.read_file(...) also accepts a file path.
• Don't forget to define the geometry column and the CRS.
• for path in Path(search_path).glob("**/*.json") is a more concise way to iterate through all JSON files within the current and sub directories. It makes the code more readable but requires at least Python 3.5.
• The example above uses a f-String which was introduced with Python 3.6.

---

However, if your Python version is older than 3.5. Here an example using os.walk(...) (and without using a f-string):

import os
import geopandas as gpd

event_tracks = gpd.GeoDataFrame(
    columns=["geometry", "eventname"],
    geometry="geometry",
    crs="EPSG:4326"  # replace with correct CRS
)

search_path = os.getcwd()

for root_path, _, file_names in os.walk(search_path):
    
    for file_name in file_names:
        
        if file_name.endswith(".json"):

            print("Reading Track Path JSON " + file_name)
    
            file_path = os.path.join(root_path, file_name)

            tracks = gpd.read_file(file_path)
            
            event_name = file_name.split(".")[0]
            tracks["eventname"] = event_name
            
            event_tracks = tracks.append(event_tracks)