0

I am currently running an experiment on automating a task by merging information from various file types and doing a number of operations to achieve this.

Currently, I am reading a list of GeoJSONs and trying to build a GeoDataFrame that contains both the geometry and the filename.

Example of file name structure in folder is: event1.json

The desired geodataframe should look like this:

    index     geometry                      event_name
    0         LINESTRING1(coordinates...)   event1
    1         LINESTRING2(coordinates...)   event2
    2         LINESTRING3(coordinates...)   event3

I am able to read the JSONs and create the geometries, but I am not able to assign the matching filename.

     import os, geopandas as gpd

     event_tracks = gpd.GeoDataFrame(columns=['geometry', 'eventname'])

     directory = os.getcwd()

     
     for dir_root, dir_dir, js_file  in os.walk(directory):
       for f in js_file:
         if f.endswith(".json"):
         fname = os.path.basename(f)
         fname, fext = fname.split('.')
         print (fname)
         print ('Reading Track Path JSON ' + f)
         with open(f, 'r') as path_f:
           event_tracks = gpd.read_file(path_f).append(event_tracks)

The code above works for geometries and generates a geodataframe with valid geometries. However, I cannot assign the event name into the column. So far, I've tried to:

  • Create list of names and append list elements
  • Reindex and concatenate two different dataframes

However, it either returns a NaN-filled column or an error such as: TypeError: cannot concatenate object of type '<class 'list'>'; only Series and DataFrame objs are valid.

Is there any workaround to do this?

0

1 Answer 1

1

Assuming that the event name is equals the file name, you would need to assign the event name to each newly "imported" dataframe before appending.

import os
from pathlib import Path
import geopandas as gpd

event_tracks = gpd.GeoDataFrame(
    columns=["geometry", "eventname"],
    geometry="geometry",
    crs="EPSG:4326"  # replace with correct CRS
)

search_path = os.getcwd()

for path in Path(search_path).glob("**/*.json"):
    
    print(f"Reading Track Path JSON {path.name}")

    tracks = gpd.read_file(path.absolute())
    tracks["eventname"] = path.stem  # file name
    
    event_tracks = tracks.append(event_tracks)

Some other notes:

  • There is no need to open the file since gpd.read_file(...) also accepts a file path.
  • Don't forget to define the geometry column and the CRS.
  • for path in Path(search_path).glob("**/*.json") is a more concise way to iterate through all JSON files within the current and sub directories. It makes the code more readable but requires at least Python 3.5.
  • The example above uses a f-String which was introduced with Python 3.6.

However, if your Python version is older than 3.5. Here an example using os.walk(...) (and without using a f-string):

import os
import geopandas as gpd

event_tracks = gpd.GeoDataFrame(
    columns=["geometry", "eventname"],
    geometry="geometry",
    crs="EPSG:4326"  # replace with correct CRS
)

search_path = os.getcwd()

for root_path, _, file_names in os.walk(search_path):
    
    for file_name in file_names:
        
        if file_name.endswith(".json"):

            print("Reading Track Path JSON " + file_name)
    
            file_path = os.path.join(root_path, file_name)

            tracks = gpd.read_file(file_path)
            
            event_name = file_name.split(".")[0]
            tracks["eventname"] = event_name
            
            event_tracks = tracks.append(event_tracks)
3
  • Thank you! I only got to look at it today, but it worked. I guess when you spend to much time on it, you fail to see the issue. For clarity, I used the first solution, for my version of Python 3.7.
    – crisvr
    Feb 7 at 15:16
  • You are welcome. I'm glad I was able to help you. If my answer solved your question, please select it as accepted answer.
    – Thomas
    Feb 10 at 16:24
  • Done. Sorry, I thought I ticked that mark.
    – crisvr
    Feb 10 at 17:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.