Why does arcpy.ListFeatureClasses() take so incredibly long?

Question

I'm having an issue where calling arcpy.ListFeatureClasses() in a workspace with 400 shapefiles takes a very long time (more than 10 minutes).

Using the os module to achieve the same effect takes less than a second:

import os
allSingleNonRoadPolys = []
for file in os.listdir(r'D:\data\allSingleNonRoadPolys'):
if file.endswith(".shp"):
    print(file)
    allSingleNonRoadPolys.append(file)

You can then achieve the same effect as ListFeatureClasses by simply prepending the workspace to the filename, each time you need to use a dataset, as seen here:

for shp in allSingleNonRoadPolys:
    arcpy.someTool(workspace + '\\' + shp)

Why would I even keep using ListFeatureClasses() and what makes it take so long? Does it do anything incredibly useful that cannot be achieved in any other way?

Answer 1

Are you sure it is arcpy.ListFeatureClasses() that takes such a long time? Could it be some other piece of code? Verify with the profiler with just a dummy os.time as shown here.

On the SSD disk (2 years old, was heavily used daily), the arcpy.ListFeatureClasses() returns the list of ~800 shapefiles found in the folder specified in less than 5 secs (just tested). The total size of the folder is ~6GB. I thought that if you have large shapefiles (more than 100MB each and your folder is 40GB), then it could have slown down the run. Yet the run time is the same for a folder of ~40GB with a hundred of shapefiles and the performance was identical to what I've observed with the larger number of shapefiles of smaller size.

Remember that you can limit your search by name, feature type, and optional feature dataset. This is helpful if you don't need to list all of your feature classes (shapefiles).

Why would I even keep using ListFeatureClasses()

You would need to use arcpy.ListFeatureClasses() to work through any Esri workspace (such as a file or a multiuser geodatabase). Of course you are free to use any other methods for listing your shapefiles (as this is an open file format to interact with), but you will need to get back to this function when working with anything stored in an Esri repository later on.