I've been trying to create a contour map of a DEM and I've been running into some technical problems. I want to avoid artifacts, so I'm using a DEM of the entire area. This means the size is rather large, over 40 gb. I then run gdal_contour to get a shapefile.
I think gdal started throwing out errors when the .shp file reached over 4gb. This is over the filesize limit of 2gb, so I did not get a finished map or valid shapefile. So I was wondering if anyone had experience working with large raster sets, and had suggestions for an appropriate file format.
Other considerations:
Space is a small consideration. I created a few contour maps of a small sample section, and noticed that the other formats were over two and a half times the size of the shapefile.
Ram is an issue. I'd like to keep ram usage minimized. I only have 16gb of ram, and I saw gdal_contour go up to 12gb of ram used. My fear is that going to a more space intensive format will increase ram usage. I can go up to 32gb of ram, but I'm also wanting to create another contour map with a higher definition, which will increase ram usage.