Given a larger csv (1000+ points) with attributes, I need to do both attribute and spatial queries on the data. Would it be faster to read the points into an xy feature class, and do the spatial and data queries as table views and feature layers, or would it be faster to read the csv into a temporary SQLite3 database, do the data queries, then transform the output into an xy class to do the spatial queries?

Also, would it be faster one way or the other if we also needed to include table joins? Lastly, if you have any alternate methods please add them.

  • Apologies for the double post... – om_henners Jul 26 '10 at 13:05
  • In reply to @George: we have a series of CSV files, some with station locations and IDs, others with test results over time for air quality, rainfall etc. The idea being that we aggregate and join some of these test results to our points, and select various points based on areas of interest. I think you're right on the spatial query side, I'm really most interested in the best/fastest way to aggregate the data and query it afterwards. Cheers, -Henry – om_henners Jul 26 '10 at 13:43

I'm pretty sure that it would be easier to use the pre-existent features of ArcGIS to do your tasks. Querying a sqlite3 database is easy, but not as easy as using the search cursor in python arc objects API.

The need for table joins can complicate the ESRI side of things. Are your "to-be joined data" stored in what DBMS?

Spatial operations will always be easier to use ESRI API. The only difference would happen if you would use something like GeoScript, that has some spatial features or other geometry framework.

Give us more information and let's see if we can help out.

  • Thanks George. Eventually did it by joining and flattening the tables in sqlite, and then doing the spatial stats using the objects api. – om_henners Jul 27 '10 at 10:49
  • Glad to hear that! – George Silva Jul 27 '10 at 12:24

One alternative would be an InMemoryWorkspace, which can also be used from python.

There are examples in the old esri forums.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.