I've recently joined an organisation where we use ArcGIS to do a lot of non-spatial analysis on a spatial dataset. Our main dataset is an environmental polygon survey with lots of attributes (in an ESRI file geodatabase), and I'm frequently asked to conduct analyses such as: - Calculate the summed area of all polygons which meet certain queries (specified by a Where clause across several fields, for example)
My problem is that I don't know very much at all about traditional database development, but it strikes me that all I'm doing is querying the database (across multiple fields) and then summarizing the results. I've been doing this is in ArcGIS and Python using tools search as Summary Statistics or objects such as searchcursors to scroll through my layers, select records and sum them.
But I imagine non-spatial databases must be a lot faster. But which one? Please tell me what platform I should use to conduct routine summaries on a 500k records sized database. MS Access? Excel and pivot tables? SQL? I don't know where to start, but have a feeling that doing this in ArcGIS (v 10, not 10.1) isn't the best way.
I'm not looking for long answers, just a short shove in the right direction to fill in my lack of knowledge about databases in general.