2

I am using Python and ArcGIS 10.1.

Is there a function or very quick way to extract points out of point feature class into an array without looping through a search cursor (as I have millions of points to process).

I eventually want to transform this into a multipoint feature and make a convex hull from it.

UPDATE:

Although, initially I thought the 11 million point convex hull optimisation was good, I think it is really too long, in order to create a random convex hull and then do an intersect and switch selection, it took 4 hours and 30 mins. That's NOT INCLUDING the actual processing of the sub-selection of points to (some 100K) create the final convex hull result. So I think it may be back to the drawing board.

The real success here for actually completing long operations and memory intensive ones, has been to upgrade ArcGIS Python geoprocessing to 64bit. Before I did this, it would just hang, now it has all the memory it needs and can really slog through some long iterative processes.

Improve this question
10
  • Whats grouping the points? Is it a polygon layer? If so and they are non-overlapping you could do a spatial join to pass over the polygon ID to the points?
    – Hornbydd
    Commented May 8, 2013 at 13:45
  • No - I am creating a convex hull FROM the points, but first you must create a mulitpoint first, then call the convexHull() function.
    – Vidar
    Commented May 8, 2013 at 14:07
  • It's not clear to me precisely what the performance issue is, but assuming it would be much faster to perform a spatial query of the points rather than loop through them, you could adopt a standard method of speeding up convex hull calculations: fetch a small number of points randomly (100 will more than suffice), compute their convex hull, then issue a query for all points lying on or outside this hull. Except in unusual cases (e.g., all points lie along the perimeter of a convex shape), this results in a huge decrease in the number of points to process.
    – whuber
    Commented May 8, 2013 at 15:03
  • @whuber - the performance issue is: looping through ALL the points in the search cursor to put them into an array in order to make a multipoint out of them - that takes a crazy amount of time!
    – Vidar
    Commented May 8, 2013 at 16:32
  • 1
    @whuber - I have put your idea into Python code now and it seems to have worked excellently for a feature class of 11 million points - it cut down the processing to 750K points and the convex hull was perfect. Now doing a 16 million one, but alas no luck yet - I have applied a new spatial index in the hope it will speed things up.
    – Vidar
    Commented May 10, 2013 at 10:59

2 Answers 2

Reset to default
1

Why can't you just the the 'Minimum Bounding Geometry (Data Management)' tool in the toolbox?

There is a group field and there would be no need to loop through each dataset. Point data is really easy to work with, but if you are trying to push your feature geometries into memory and the larger case occurs you can hit a memory errors. The List object in python will also hit performance issues when it gets really huge.

Improve this answer
0

If you data is stored in a database SQL server postgres etc. you can use pyodbc to connect to the database and the use cursor.execute(SQL) to do queries, and write to a csv.

Improve this answer
3
  • No - using Oracle. That's an option, to use the database side - but I'm just working through Python at the moment.
    – Vidar
    Commented May 8, 2013 at 14:08
  • 1
    Worth mentioning- In arcgis 10.1 (I am assuming you are using this) arcpy has the da module which has cursors that are apparently a lot faster than the standard arcpy cursor.
    – Hornbydd
    Commented May 8, 2013 at 22:57
  • @Hornbydd - I didn't actually know that, but thanks, I will look into that.
    – Vidar
    Commented May 10, 2013 at 10:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.