I'm a novice GISer who needs to do some basic computations on very large (say ~20GB) files. Nothing complicated, merge, intersect, add XY columns kind of thing. It's a necessary step for a project, but it won't be something I do frequently. From what I understand, ArcMap refuses to work with files over 2GB and I think QGIS would crash with these files.
As I can see it, my options are:
- Knuckle down and learn GIS python scripting just for this one computation
- Break my computation down into 2GB chunks and run it 10 times, hoping none of the intermediates exceed 2GB.
- Find software or file formats that can handle large files.
Number 3 would be the easiest if such software exists but I'm not sure it does. If not, do I have any options other than 1 and 2? Both are unpalatable (being new at scripting for 1, tedium and possibility that it doesn't work for 2). Are there other options?