Combining multiple shapefiles in arcgisscripting?

Question

I have got a lot of point shapefiles (r20010101.shp, r20010202.shp, r20020101.shp...). Each of them has the same amount of points at exactly the same location. In a column that is named after the Date e.g. "D20010101", where I have got unique values for every shapefile.

What I want to do now, is to Combine or Merge or Join all these shapefiles in a Python command.

The result should be ONE shapefile with all those point (but please NO duplicate ones!!).

The attribute table of this shapefile should look like this: FID, Shape, Id, [Value of Date 2001/01/01], [Value of Date 2001/02/02],...

Answer 1

NOTE: This works in ArcMap10. You will have to test it in 9.3. -- Join the three shapefiles together using the Join tool. Make sure you have an indexed feature that can be used to join them on (it speeds the tool up dramatically). After doing that, you can use the Fields tab inside the shapefile / layer properties to make certain fields visible/hidden. If you hide fields and export the layer to a new feature class / shapefile, the resulting file will only have the visible fields.

Alternatively, once you have the join, you can export it wholesale and perform any number of tools upon it:

1. Use merge and mangle the field map (allows renaming of fields). Yes, you can perform a merge on a single layer just to change names and whatnot.
2. Use ModelBuilder to create a batch Field Delete python script for your shapefiles.
3. Add Field to get new names and calclulate the fields you want with the old data. Then you can delete the old fields with their ugly qualified names.

Last, and least (slow process is slow), you could create new fields and cursor through your data, reading the date information from the other tables row by row.

So, as you can see, many options! And all of it is accessible through the GP processor (although I find joining multiple layers at once to be unreliable in a script).

Answer 2

This is what I was looking for.

import arcgisscripting, os
import sys

gp = arcgisscripting.create(9.3)
gp.Workspace = "D:\\Workspace\\shapedata"
# With Files:
# r20010101.shp with Fields and Input like:
# FID,Shape,POINTID,D20010101
# 0, Shape, 1, 10
# 1, Shape, 2, 14

# r20020101.shp with Fields and Input like:
# FID,Shape,POINTID,D20020101
# 0, Shape, 1, 12
# 1, Shape, 2, 13

# r20020601.shp with Fields and Input like:
# FID,Shape,POINTID,D20020601
# 0, Shape, 1, 11
# 1, Shape, 2, 9

# r20030101.shp with Fields and Input like:
# FID,Shape,POINTID,D20030101
# 0, Shape, 1, 3
# 1, Shape, 2, 7

fcs = gp.ListFeatureClasses("", "POINT")

x=0
for fc in fcs:
    outfc = "D:\\Workspace\\shapeout"
if x==0:
    # just run once:
    x=x+1
    fc_old=fc #r20010101.shp
    fn_old="D"+fc[1:-4] #D20010101
else:
    # do this, as long as there are shapefiles in gp.workspace
    # examples given here for the FIRST SpatialJoin

    fc_new=fc # r20020101.shp
    outfc=outfc+str(x) # D:\\Workspace\\shapeout1
    fn_new="D"+fc[1:-4] #  D20020101

    fieldmappings = gp.CreateObject("FieldMappings")
    vt = gp.CreateObject("ValueTable")

    fieldmappings.AddTable(fc_old)
    fieldmappings.AddTable(fc_new)

    if x>1:
        # This should prevent Fields "Join_Count", "Join_Co1", "Join_Co2"....
        fieldmappings.removeFieldMap(fieldmappings.findFieldMapIndex("Join_Count"))

    gp.SpatialJoin(fc_old, fc_new, outfc, "#", "#", fieldmappings) 

    if x>1:
        # delete temporary file e.g. shapeout1.shp after next loop 
        gp.delete_management(infc)

    fc_old=outfc + ".shp" # D:\\Workspace\\shapeout1.shp will be joined in next loop with e.g. r20020601.shp

    x=x+1

# The final Output will be 1 Shapefile with Fields and Input like: 
# FID,Shape,POINTID,D20010101,D20020101,D20020601,D20030101
# 0, Shape, 1, 10, 12, 11, 3
# 1, Shape, 2, 14, 13, 9, 7

As I have got 30 Pointfiles (150MB each), this takes pretty long on my Computer (not too old), like 1-2 hour for every SpatialJoin!

Answer 3

if the "ID" field is unique within each table and consistent for all point across the many shapefiles, you can just use the join field tool under Data Management - Joins - Join Field.

as Nathanus stated, make sure the ID field of each shapefile is indexed.