1

I am reading a feature class into a pandas dataframe using:

df = DataFrame(
    arcpy.da.FeatureClassToNumPyArray(
        in_table=feature_class,
        field_names=field_list,
        skip_nulls=False,
        null_value=np.NaN
    )
)

Where feature_class is the name of the feature class and field_list is the list of fields to include.

Setting skip_nulls=False ensures the floats are converted to np.NAN and setting null_value=np.NAN does the same for integers. However, one of the columns has a date type, which causes the following error:

ValueError: year 0 is out of range

Is there any way to set a specific default for the date type?

  • 2
    Have you tried without numpy: df = pd.DataFrame.from_records(data=arcpy.da.SearchCursor(feature_class, field_list),columns=field_list) – BERA Jul 9 at 13:49
  • 2
    That actually works pretty well! Thanks @BERA – Pau Jul 10 at 6:57
1

Building on @BERA's solution, a more robust function to read tables/feature classes into dataframes would be:

def to_dataframe(data, field_list=None, exclude=None):
    """
    load data into a pandas data frame for subsequent analysis.
    :param data: input arcgis feature class or table.
    :param field_list: fields to be used.
    :return: pandas dataframe object.
    """
    # if empty, get field list from data
    if field_list == None:
        fields = arcpy.listfields(data)
        field_list = [f.name for f in fields]

    # Build dataframe
    df = pd.DataFrame.from_records(
            data=arcpy.da.SearchCursor(data, field_list),
            columns=field_list,
            exclude=exclude
            )

    return df

This method coerces dates, and also turns the SHAPE field into a tuple of points.

  • you should mark your own answer as the answer so other can see a solution was found. – Hornbydd Aug 10 at 12:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.