4

I am collecting start and end points for a large polyline feature class in a file geodatabase. I am using arcpy, ArcGIS 10.3, Python 2.7.3 64-bit and a 10.3 compressed file geodatabase with just one feature class (and no other objects).

One accesses the geometry properties with the SHAPE@ token which gives access to the geometry of the line (namely firstPoint and lastPoint).

with arcpy.da.SearchCursor(fc, ["OID@", "SHAPE@"],query) as search_cur:
        allLinesList = [[row[0],(row[1].firstPoint.X,row[1].firstPoint.Y),
                         (row[1].lastPoint.X,row[1].lastPoint.Y)]
                            for row in search_cur]

Reading 1 million line features (note that my lines consist of only two vertices, start and end) takes ca 5 minutes. I have tried using for loop iterating a cursor instead of comprehension and iterating features in chunks of several hundred thousands with no significant difference in time.

Do you know any other faster way to read XY of the start and end points of the lines?

  • Do you need to use a compressed geodatabase? – PolyGeo Feb 27 '15 at 11:34
  • @PolyGeo, no I don't - compressed file geodatabase might provide faster read-only access that is why I am compressing it – Alex Tereshenkov Feb 27 '15 at 11:46
  • I was thinking the opposite but have not tested to see which is faster. – PolyGeo Feb 27 '15 at 11:57
  • I'm on iPhone so going from memory but if lines all have just start and end points and no intervening vertices could you convert them to points and then use a tool to add XY fields and read those fields any quicker overall? – PolyGeo Feb 27 '15 at 12:18
  • 3
    300 seconds doesn't seem outrageously long to access 1m lines through a cursor. I would suggest you try with an uncompressed FGDB as well. On what kind of disk is the file? – Vince Feb 27 '15 at 12:56
1

I suggest you to write a proxy class in managed C++ if this is an option for your.

Realistically, you might need to perform the kind of operation that takes longer in .NET or Arcpy(large iterations of small operations).You have the choice to write a proxy class in Managed C++ where you can place ArcObjects code that heavily uses the interop layer.

Once you expose this code as public methods, you can call the methods from your C# or VB .NET and even from arcpy using comtypes. Since the interop layer is only required when you call the methods of this proxy class, the execution time can be greatly reduced.

For example a method for creating a line feature with 1 million records take 30 seconds with .NET, but using managed C++, it takes 5 seconds (6 times faster).

Proxy class is the ultimate choice, that you can get the most out of ArcObjects. This is what arcpy data access module does to improve performance.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.