1

I am trying to take a shapefile, with way too many fields/columns, and copy over only certain fields into a new shapefile.

Normally, I would just delete the fields I don't need from the original shapefile, but I don't want to come across an instance where the GIS operator decides to remove a field that my automated .py scripts is trying to delete, causing the program to fail.

At this point, I have not physically tried anything because every tool I have come across I have been able to find documentation proving it's lack of usefulness (Fields -> Transpose Field and General -> Copy).

Transpose Field

Fields -> Transpose Field does copy individual fields but it transposes them from a column into a row appendage, which does not help.

General -> Copy copies the entire shapefile and will not let me choose only certain columns. What I want is something similar to this tool but one that will allow me to choose only certain fields to include in the copy.

I use "Feature Class To Shapefile (multiple)" to convert my features into shapefiles, and I tried looking into a "Feature to Shapefile" solo Tool that would let me pick the columns/attributes of the feature to convert. That has so far turned up empty as well.

I am using the ArcCatalog ModelBuilder to perform this task, not raw Python scripts. I know enough Python to go that route if needed, but a Toolbox Tool is the preferred goal. This task will be part of a larger whole that will be run via a batch file calling the exported Python code from the Models. It has to be something that can be run automatically, so manually copying/pasting is not applicable.

As for a screenshot of the Model, my current setup has each task as individual models: one model copies rows, one creates shapefile from feature class, one adds a field, one populates that field, etc. I'm sure this is probably inefficient but this task has been passed on to me from someone else and this was how it was done. I can add screenshots of the other models I have but I'm not sure what use that will provide, as it doesn't give much of a context.

Basically, I'm looking to do something like Copying data from one attribute field to another using ArcGIS Field Calculator? but with about 10 different fields. I don't want to do them individually, but using one or maybe two tools.

  • 1
    I know this doesn't answer your question, but in response to "causing the program to fail" I believe that arcpy.DeleteField_management() will not fail if told to delete a field that doesn't exist. – Midavalo Feb 20 '17 at 16:41
  • Your script should certainly test for the existence of a field before deleting it. – Vince Feb 20 '17 at 16:42
  • @Midavalo That actually is very helpful, as it would eliminate the only reason for this need. Although, I still would like to know if there is a way to do it without deleting, in the case that I need it in the future. – Hank Feb 20 '17 at 16:42
  • 2
    There are always multiple ways to do anything in GIS. Unchecking the columns in a layer then saving the data would much faster than multiple delete operations. Please edit the question to specify what you have tried. – Vince Feb 20 '17 at 16:45
  • 1
    @Hank no problem - Have a go trying a few tools in ModelBuilder and update your question to include the result (and how that does or doesn't work for you) and include any code snippets that produces if you want to go down the arcpy path (if necessary). Seeing what you've tried helps potential answerers not suggest something you've already tried, and also can help them fully understand what you are trying to do – Midavalo Feb 20 '17 at 16:53
4

You can take advantage of using a FieldMappings parameter type. There are many tools that support this, you probably want to use the Feature Class To Feature Class tool (which also works on shapefiles). You can alter the field map there for use in Model Builder.

If you are using Python, you can pass in a list of fields that you want to keep like this:

import arcpy
import os

def copy_with_fields(in_fc, out_fc, keep_fields, where=''):
    """
    Required:
        in_fc -- input feature class
        out_fc -- output feature class
        keep_fields -- names of fields to keep in output

    Optional:
        where -- optional where clause to filter records
    """
    fmap = arcpy.FieldMappings()
    fmap.addTable(in_fc)

    # get all fields
    fields = {f.name: f for f in arcpy.ListFields(in_fc)}

    # clean up field map
    for fname, fld in fields.iteritems():
        if fld.type not in ('OID', 'Geometry') and 'shape' not in fname.lower():
            if fname not in keep_fields:
                fmap.removeFieldMap(fmap.findFieldMapIndex(fname))

    # copy features
    path, name = os.path.split(out_fc)
    arcpy.conversion.FeatureClassToFeatureClass(in_fc, path, name, where, fmap)
    return out_fc

if __name__ == '__main__':

    fc = r'C:\TEMP\test2.gdb\Rental_Licenses'
    new = r'C:\TEMP\test2.gdb\Rental_Licenses_filtered'
    fields = ['Match_addr', 'PROPERTY_O', 'OWNER_ADDR'] #only these fields and other required fields will be included
    copy_with_fields(fc, new, fields)

I know you can also use the DeleteField tool, but I have found that to be very slow in tables that have lots of fields.

  • I have the DeleteField as an option and, as Midalvo pointed out, it would not fail like I had initially worried, but I am still curious if the Delete route can be avoided at all. – Hank Feb 20 '17 at 17:27
  • This does avoid the deleting route, it just only outputs the fields you want to keep. And in this case, you can simply pass in a list of fields you want to keep for the shapefiles. You can accomplish the same thing in model builder too, as the Feature Class To Feature Class tool (works on shapefiles too) supports the field mapping parameter. – crmackey Feb 20 '17 at 17:27
  • Is the "Feature Class to Feature Class" tool inside of Conversion? – Hank Feb 20 '17 at 17:28
  • Yes in the "To Geodatabase" section, but it will work for creating shapefiles as well. – crmackey Feb 20 '17 at 17:29
  • 2
    That did exactly what I wanted. I was looking for that type of tool but didn't think to look inside of "To GeoDatabase". If you will update the answer to include the suggestion in the comments, I will mark it as the answer. I want others to be able to find the solution easier and not require them to read the comments :) – Hank Feb 20 '17 at 17:47
3

To avoid the need for scripting and to do this quickly do the following: Open ArcMap and add the shapefile to a blank or existing map. Open the layer properties of the shapefile (double click the layer or right click the layer and choose properties), go to the Fields tab, turn off the visibility of all fields (the second button above the field list outlined in red below) and then turn on the visibility of just the fields you need by checking the box next to each field name.

enter image description here

At this point all the fields will be hidden in the tableview of the layer, which may be all you need if you don't really have to actually delete all of the fields, just hide them in a particular map.

To fully remove the fields right click the layer in the map's Table of Contents (the layer list) and choose Data-> Export Data..., set the output location and shapefile name, and press the OK button. This will copy the layer features to a new shapefile with just the fields you want.

ModelBuilder can do the same thing as the layer properties using the Make Feature Layer tool and its field list and all other tools that take a layer as input will respect the field list whether or not the tool that takes that input supports a field list.

  • While this is a good option normally, I am operating on data that belongs to a county that isn't mine. I cannot change their original data, just take it and copy it into what I want. I also cannot rely on anything to be done manually so it all has to be done with a script that can be called by a batch file run on a schedule. – Hank Feb 20 '17 at 18:36
  • The approach is still valid in ModelBuilder. I have adjusted my comments to show that what you can do in ArcMap applies to layers in ModelBuilder as well. – Richard Fairhurst Feb 20 '17 at 18:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.