0

I have a txt that contains in each line a sentence like this:

INT_ASTIK_EAS
ER_ASTOM_ASTIK
and so on.

I want to ask for a directory of the main folder that contains the subfolders that each contain the ASTIK.shp, then the EAS.shp, then the ASTOM.shp, etc

INT means to perform an intersection with this code where in this case should involve ASTIK and EAS shapefiles.

inte_s=gpd.overlay(ASTIK,EAS,how='intersection')

the code:

rootdir = r'C:\Users\user\Desktop\a' # path to your root directory you walk
sfiles = {} # a dictionary with all the .shp files
for entry in os.listdir(rootdir):
    dirpath = os.path.join(rootdir, entry)
    if os.path.isdir(dirpath): 
        for file in os.listdir(dirpath): # Get all files in the subdirectories
            if file.endswith('.shp'): # If it's an .shp.
                filepath = os.path.join(dirpath, file)
                sfiles[filepath] = gpd.read_file(filepath)


action_dict = {'INT': 'intersection', 'ER': 'difference'}


directory=input('Insert dir of the main folder')
with open(input()) as txtfile: #insert directory of txt
    x = txtfile.readlines()
for line in x:
    action, shape1, shape2 = line.split('_')  # here line is ER_ASTOM_ASTIK or whatever line in the txt file
    if shape1 in sfiles and shape2 in sfiles:
        gpd.overlay(sfiles[shape1], sfiles[shape2], how=action_dict[action])

It doesn't probably perform the intersection or difference. How can i store its results in a variable because i have to do further process with the outcomes of every operation of every line.

After checking what's the issue with the sfiles variable and what it stores: it gives this:

for x in sfiles:
    print (x)

C:\Users\user\Desktop\a\ASTTOM\ASTTOM.shp
C:\Users\user\Desktop\a\PST\PST.shp

and finally:

if shape1 in sfiles:
    print('success')
else:
    print('failed')

failed

which means it doesn't communicate what it should in the dictionary because the shapefiles happen to exist in the folder. How can i make it do the overlay calculations?

print(sfiles)

{'C:\\Users\\user\\Desktop\\a\\ASTTOM\\ASTTOM.shp':      OBJECTID  FID_Asttom TheOTA_Tmp  aa    SHAPE_Leng    SHAPE_Area  \
0           1           1      20008   7   4206.346569  5.796467e+05   
1           2           2      20008  11   3311.521073  6.892735e+05   
2           3           3      20008   6   2131.153007  2.196689e+05   
3           4           4      20008   2   1738.249605  4.731495e+04   
4           5           5      20008   1   1075.903789  5.337986e+04   
5           6           6      20008  10   3255.422426  3.628100e+05   
6           7           7      20008   3   2469.427102  2.267493e+05   
7           8           8      20008   5   2466.593786  2.042149e+05   
8           9           9      20008   8   3095.473108  5.384131e+05   
9          10          10      20008   4   1856.659243  1.136976e+05   
10         11          11      20008   9   2293.865083  2.063858e+05   
11         12          12      20104   4   4445.338873  4.455547e+05   
12         13          13      20104   7   3659.621424  2.834771e+05   
13         14          14      20104   2   2098.126160  1.981298e+05   

it prints the attribute tables of all the shps inside the dictionary

if attempt this:

print(sfiles[shape1], sfiles[shape2], action_dict[action])

KeyError                                  Traceback (most recent call last)
<ipython-input-11-9f435f40293e> in <module>()
----> 1 print(sfiles[shape1], sfiles[shape2], action_dict[action])

KeyError: 'ASTTOM'


and each every time:
print(sfiles[shape1])
KeyError: 'ASTTOM'

print(sfiles[shape2])
KeyError: 'PST'

print(action_dict[action])
difference

What it gets printed with the new code:

print(group)
['C:\\Users\\user\\Desktop\\a\\ASTTOM\\ASTTOM.shp']

print(grouped_shapefiles)
[['C:\\Users\\user\\Desktop\\a\\ASTTOM\\ASTTOM.shp'], ['C:\\Users\\user\\Desktop\\a\\PST\\PST.shp']]

print(command)
INT_ASTTOM_PST


print(commands)
['INT_ASTTOM_PST', 'ER_ASTTOM_PST']

command.split('_')
['INT', 'ASTIK', 'EAS']

and finally df1 prints the attribute table of the first shp

4
  • 1
    just a quick guess, but did you try gpd reading WITH the .shp extension in your file names
    – radouxju
    Jun 28, 2018 at 8:00
  • @radouxju I thought you found the problem but after testing it it still won't fix the issue it even read the file without it. I updated the answer with what the sfiles returns. Jun 28, 2018 at 8:12
  • Looks like it. There are about 50 operations but since they are gonna be added more i thought reading from the txt would be beneficial.What do you suggest? Jun 28, 2018 at 8:49
  • about their outputs it only needs to check if there are any rows left after the operation. Jun 28, 2018 at 9:19

1 Answer 1

1

It is still unclear exactly what you are asking, for example what output do you want - one for each row in the textfile or are the rows steps in an algorithm to produce one output? .

I think your problems are in the way you are listing paths to shapefiles and using dictionaries. Try:

import os
import geopandas as gpd

commandfile = open("C:\Test\commands.txt","r")
rootfolder = r'C:\Test\Root_folder'

def splitpath(path):
    folders = []
    while 1:
        path, folder = os.path.split(path)

        if folder != "":
            folders.append(folder)
        else:
            if path != "":
                folders.append(path)

            break
    folders.reverse()
    return folders

#List all shapefiles as full path and filename
shapelist = []
for path, subdirs, files in os.walk(rootfolder):
    for name in files:
        if name.endswith('.shp'):
            shapelist.append(os.path.join(path, name))

#List all subfolders
subfolders = set(map(lambda x:splitpath(x)[-3], shapelist))
#Create list of sublists where each sublist are the shapefiles in that subfolder
grouped_shapefiles = [[y for y in shapelist if splitpath(y)[-3]==x] for x in subfolders]

#List commands
commands = commandfile.read().splitlines()

action_dict = {'INT': 'intersection', 'ER': 'difference'}

results = []
for group in grouped_shapefiles:
    for command in commands:
        #print(command)
        df1 = gpd.read_file([g for g in group if command.split('_')[1] in g][0])
        df2 = gpd.read_file([g for g in group if command.split('_')[2] in g][0])
        df = gpd.overlay(df1,df2,how=action_dict[command.split('_')[0]])
        results.append(df)
#Do something will the results...

enter image description here

It is important the folder structure is exactly like this (rootfolder\subfolder (which is used to group together shapefiles to overlay)\shapefolder1,2..n) or the code will have to be modified: enter image description here

2
  • To make it clear about the output.I need the output of every operation to be checked for some things.For example in the first intersection i will check the output if it contains any rows. Things like that but not always this specifically. Jun 28, 2018 at 10:28
  • Every single shapefile listed in the txt is in one of the subfolders. Jun 28, 2018 at 12:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.