Our organization is considering moving our geoprocessing workflow to PostGIS. We are currently using ArcGIS, with a plethora of custom Python tools used in ModelBuilder. We're moving most of our data into PostGIS to be consumed by a variety of apps, and now we're asking if it also makes sense to perform the data processing there as well.
We process data to be compatible with our software. A customer purchases our software, gives us their data, and we process it to be optimized for use in our software. This requires us to build a variety of tools to handle varying qualities of input data. We can't expect to receive data in a particular format or schema, so we build tools to map input fields to output fields, parse single fields into multiple fields, merge multiple datasets, etc. We also perform spatial joins, intersections, trim whitespace and concatenate fields, and many other common operations. PostGIS appears to be perfectly capable of performing all of our processing needs.
For those of you who use PostGIS to do your data processing, do you have any advice for organization, tools to use, etc.?
- do you use it in conjunction with QGIS python processing?
- are people using a Python ORM for non-web processing? I've been leaning towards using GeoDjango since it has a Python ORM for PostGIS. Our initial test of using PostGIS to process data has many large SQL text blocks in Python code and we are thinking that the GeoDjango ORM may help with creating more manageable and readable code. There's also the GeoAlchemy ORM that interacts similarly with PostGIS, and doesn't appear to be as web-specific as Django is.
I haven't heard of people using PostGIS to do geoprocessing as much as I see people using QGIS or ArcGIS, so I want to know if it is a comparable alternative.