Reputation
3,261
Top tag
Next privilege 5,000 Rep.
Approve tag wiki edits
Badges
12 30
Newest
 Good Answer
Impact
~127k people reached

May
14
revised Methods for converting a regular polygon grid to a numpy array
deleted 100 characters in body
May
14
revised Methods for converting a regular polygon grid to a numpy array
Updated answer to feature code that explicitly addresses the question - converting a grid to a numpy array.
May
14
revised Methods for converting a regular polygon grid to a numpy array
Added an updated piece of code related to converting a polygon grid to a numpy array.; deleted 1386 characters in body
May
14
awarded  Good Answer
May
13
revised Efficient ways to read table data from a large feature layer?
Fixed minor error in code.
May
13
accepted Efficient ways to read table data from a large feature layer?
May
13
answered Efficient ways to read table data from a large feature layer?
May
13
revised Methods for converting a regular polygon grid to a numpy array
Updated the lines of code following the table to numpy conversion to a different method that utilizes numpy properly.
May
12
comment Efficient ways to read table data from a large feature layer?
@MichaelMiles-Stimson, the table is about 70 MB with around 460,000 rows so it's not huge. I tried using arcpy.CopyFeatures_management() to an in_memory location, but that copy takes about 40 minutes which doubles the time to process even though the TableToNumpyArray on the resulting table now takes 0.2 seconds.
May
12
revised Efficient ways to read table data from a large feature layer?
Replaced the words "process" and "extract" with "read" in body text for precision.
May
12
comment Efficient ways to read table data from a large feature layer?
@DanPatterson, I copied the joined features to disk and TableToNumpyArray (T2NP) ran in 6 seconds - huge improvement there! However, it took 40 minutes (twice as long as T2NP) to copy the features to disk...
May
12
comment Efficient ways to read table data from a large feature layer?
Thanks for the suggestions, Alex. The reason I'm surprised at the length of time it takes to read the table (~20 minutes) is because I can run zonal statistics on the dataset in ~20 seconds. I will have to look into getting an SSD for my system!
May
12
revised Efficient ways to read table data from a large feature layer?
Edited title, replaced "extract" with "read" for precision.
May
11
comment Efficient ways to read table data from a large feature layer?
@MichaelMiles-Stimson, I'm using a file GDB - I don't think arcpy in 10.2 supports accessing personal GDBs with background geoprocessing on. There's gotta be a better solution - I really need this to be an automated process where I don't need to intervene and create intermediary files.
May
11
comment Efficient ways to read table data from a large feature layer?
@MichaelMiles-Stimson, I was thinking a non-ESRI alternative might be the only way, however these data are stored in an ESRI geodatabase and I'm using arcpy to manipulate into a feature layer in memory. I suppose I could write this out to a shapefile and read it using a different set of tools.
May
11
comment Efficient ways to read table data from a large feature layer?
@DanPatterson, I've added sample code to the question...I'm really just interested in finding a better way to read the data out of my feature layer than using the TableToNumpyArray tool in the DataAccess module of arcpy. I've tried using a search cursor to read row by row and make a list, but that appears to be equally as arduous if not more so.
May
11
revised Efficient ways to read table data from a large feature layer?
Added sample code.
May
11
asked Efficient ways to read table data from a large feature layer?
Apr
27
awarded  Popular Question
Apr
8
awarded  Yearling