3

situation

We have a folder containing 1 sub-folder for every participant in a study. The subfolders are named with the participants' ID, e.g., 1000, 1001, 1002, 1003. Every participant folder contains a shapefile (and associated files) of the same name (e.g., 1000.shp, 1001.shp, etc). Each shapefile has several points.

challenge

We need to add a column called "PID" to every shapefile. Then we need to add the participant's ID (i.e., the file name, 1000, 1001, 1002) to every row.

For instance, in the folder "1000", there is a shapefile called "1000.shp". In the associated data table, we need to add a column called "PID". Then we need to insert the value "1000" for every row in this new PID column.

question

Is there a way to automate this process?

program/languages

I work with ArcGIS, QGIS, and R. I use R extensively, but not for spatial work. That said, I might have an easier time understanding a programmatic approach in R if one exists.

  • 2
    What programming language were planning to use? Please edit your question to include this important detail. – PolyGeo Oct 2 '13 at 21:37
  • 1
    It may be possible to do with R (I don't use it myself, so I couldn't say for sure), but Python is more suitable for this use case as it is the de facto scripting language of mainstream GIS packages like QGIS and ArcGIS. For simple data management purposes, I think Python is definitely the way to go. – blah238 Oct 2 '13 at 22:18
2

The following script loops through all folders and subfolders in a workspace, adds a new field and calculates field based on your specifications.

import arcpy
import os
workspace = r"C:\your\path"

# Create an empty list which will be populated in the next step
feature_classes = []

# Loop through folders and subfolders to find featureclasses/shapefiles
for dirpath, dirnames, filenames in arcpy.da.Walk(workspace,
                                                  datatype="FeatureClass",
                                                  type="Any"):
    # Append filename to feature_classes list
    for filename in filenames:
        feature_classes.append(os.path.join(dirpath, filename))

# Loop through newly created list of shapefiles, add a new field and calculate field
for fc in feature_classes:
    field_name = "PID"
    value = os.path.basename(fc).strip(".shp") # Manipulate filename for use in calculations
    arcpy.AddField_management(fc, field_name, field_type = "LONG")
    arcpy.CalculateField_management(fc, field_name, expression = value)
|improve this answer|||||
  • 1
    Nice answer. However, I would tend to make the field_type a text field in case there are folder names that aren't numbers. – Fezter Oct 3 '13 at 3:28
2

That is pretty easy. Here is something to get you started in arcpy:

import arcpy, os

def add_filename_field(fc, fieldname):
    arcpy.AddField_management(fc, fieldname, "TEXT")
    fc_name = os.path.splitext(os.path.basename(fc))[0] # Assumes shapefile, not suitable for SDE table names
    arcpy.CalculateField_management(fc, fieldname, "'{0}'".format(fc_name), "PYTHON_9.3")

if __name__ == '__main__':
    # hardcoding these, but could also use GetParameterAsText or sys.argv
    workspace = r"C:\path_to_shapefiles"
    fieldname = "PID"

    arcpy.env.workspace = workspace
    fcs = arcpy.ListFeatureClasses()
    for fc in fcs:
        add_filename_field(fc, fieldname)

Note that this does not iterate through all the subfolders in the main folder you describe, but that is an easy enough modification that I leave it to you as an exercise.

|improve this answer|||||
1

If you want to try this in R, you have to install raster package and use this code:

install.packages("raster")
library(raster)

## Define start directory
setwd("~/Proyectos/test/")

## We save start directory for later
startwd <- getwd()

## we list all subfolder
list.subfolder <- list.dirs()

# Then for each subfolder..
for(j in 2:length(list.subfolder)){
  setwd(list.subfolder[j])
  ## ... we list all shape files
  shapes <- list.files(pattern="shp")
    for(i in 1:length(shapes)){
    ## We open every shapefile  
    temp_shape <- shapefile(shapes[i])
    ## Add a field called PID, with a substring equal to the name of the file without the ".shp" 
    temp_shape$PID <- substr(x=shapes[i], 1, nchar(shapes[i])-4)
    ## And overwrite the shapefile with the added field
    shapefile(shapes[i],overwrite=T)
    }
  ## Then we go back to the starting directory
  setwd(startwd)
  }
|improve this answer|||||

Not the answer you're looking for? Browse other questions tagged or ask your own question.