I'm starting to work with the NED 1/3 and 1/9 arc-second data. I've been using gdal in Python and QGIS to do some basic conversions of the raw .IMG data from USGS. My goal is to get elevation data of the San Francisco Bay Area that I can look up with coordinates. It seemed like the most straightforward way to do this was to use gdal to convert to .XYZ format - this resulted in a really big file from a small piece of the area I'm using as a test.
Eventually I'd like to do this for the state of California, the USA, etc. so scalability is a concern. Although I envision some sort of database, if gleaning the info directly from the IMG or some other format is better I'm open to the idea.
A similar poster reads data from the files directly - would this be a good route to go from a memory/speed point of view? What are the tradeoffs? If this is a better method than converting to raw data directly, can anyone point me to some resources on how to accomplish this in python?
What is the best way to get to where I want to be?