Timeline for Optimizing arcpy UpdateCursor on large Polygon Feature Class and Table
Current License: CC BY-SA 4.0
9 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jul 20, 2022 at 21:47 | comment | added | Vince | If it's a local FGDB, then you could search to find records (cached in a dictionary), then do one UpdateCursor scan. It might take a bit of RAM, but even 1m updates couldn't use more than 40MiB, indexing by OID. | |
| Jul 20, 2022 at 21:14 | comment | added | vitale232 |
The production data actually lives in an AGOL Hosted Feature Layer. In the script, I have everything batched into chunks of 1,000 to try and avoid REST issues, including the list comprehension SearchCursor that's in the example above. Would iterating over all of the rows in an UpdateCursor work?
|
|
| Jul 20, 2022 at 20:33 | comment | added | Vince | If the IN list is 6700+ features, the SQL query is likely to be really inefficient (Oracle won't let the number of terms exceed 2000). | |
| Jul 20, 2022 at 19:48 | answer | added | vitale232 | timeline score: 1 | |
| Jul 20, 2022 at 19:43 | comment | added | vitale232 | Thanks for the tip. I think this led me to my answer. | |
| Jul 20, 2022 at 18:58 | comment | added | vitale232 |
Interestingly, if I remove the where_clause from the UpdateCursor, I get the expected performance on the first 1,728 records. Then it starts to slow down to 2s again. Quite strange.
|
|
| Jul 20, 2022 at 18:42 | comment | added | Vince |
The IN list could be slowing down the query; the existing updateRow logic would let you go without it. You could see a 50% performance improvement if the source is switched from a network share to a local disk. If you're assigning a constant, using a selection set and a Calculate Field might be faster.
|
|
| S Jul 20, 2022 at 18:25 | review | First questions | |||
| Jul 20, 2022 at 18:46 | |||||
| S Jul 20, 2022 at 18:25 | history | asked | vitale232 | CC BY-SA 4.0 |