Take the 2-minute tour ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.

I am using Python and ArcGIS 10.1.

Is there a function or very quick way to extract points out of point feature class into an array without looping through a search cursor (as I have millions of points to process).

I eventually want to transform this into a multipoint feature and make a convex hull from it.


UPDATE:

Although, initially I thought the 11 million point convex hull optimisation was good, I think it is really too long, in order to create a random convex hull and then do an intersect and switch selection, it took 4 hours and 30 mins. That's NOT INCLUDING the actual processing of the sub-selection of points to (some 100K) create the final convex hull result. So I think it may be back to the drawing board.

The real success here for actually completing long operations and memory intensive ones, has been to upgrade ArcGIS Python geoprocessing to 64bit. Before I did this, it would just hang, now it has all the memory it needs and can really slog through some long iterative processes.

share|improve this question
    
Whats grouping the points? Is it a polygon layer? If so and they are non-overlapping you could do a spatial join to pass over the polygon ID to the points? –  Hornbydd May 8 '13 at 13:45
    
No - I am creating a convex hull FROM the points, but first you must create a mulitpoint first, then call the convexHull() function. –  Vidar May 8 '13 at 14:07
    
It's not clear to me precisely what the performance issue is, but assuming it would be much faster to perform a spatial query of the points rather than loop through them, you could adopt a standard method of speeding up convex hull calculations: fetch a small number of points randomly (100 will more than suffice), compute their convex hull, then issue a query for all points lying on or outside this hull. Except in unusual cases (e.g., all points lie along the perimeter of a convex shape), this results in a huge decrease in the number of points to process. –  whuber May 8 '13 at 15:03
    
@whuber - the performance issue is: looping through ALL the points in the search cursor to put them into an array in order to make a multipoint out of them - that takes a crazy amount of time! –  Vidar May 8 '13 at 16:32
1  
@whuber - I have put your idea into Python code now and it seems to have worked excellently for a feature class of 11 million points - it cut down the processing to 750K points and the convex hull was perfect. Now doing a 16 million one, but alas no luck yet - I have applied a new spatial index in the hope it will speed things up. –  Vidar May 10 '13 at 10:59
show 5 more comments

2 Answers 2

Why can't you just the the 'Minimum Bounding Geometry (Data Management)' tool in the toolbox?

There is a group field and there would be no need to loop through each dataset. Point data is really easy to work with, but if you are trying to push your feature geometries into memory and the larger case occurs you can hit a memory errors. The List object in python will also hit performance issues when it gets really huge.

share|improve this answer
add comment

If you data is stored in a database SQL server postgres etc. you can use pyodbc to connect to the database and the use cursor.execute(SQL) to do queries, and write to a csv.

share|improve this answer
    
No - using Oracle. That's an option, to use the database side - but I'm just working through Python at the moment. –  Vidar May 8 '13 at 14:08
1  
Worth mentioning- In arcgis 10.1 (I am assuming you are using this) arcpy has the da module which has cursors that are apparently a lot faster than the standard arcpy cursor. –  Hornbydd May 8 '13 at 22:57
    
@Hornbydd - I didn't actually know that, but thanks, I will look into that. –  Vidar May 10 '13 at 10:57
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.