2

I am new to GIS and I wish to perform view shed analysis of wind turbines from a DEM.

I have downloaded OS Terrain 50 and the data is contained within numerous zipped folders.

I am using a Mac with OS X Mavericks and I would like to know how to unzip all the data and merge them into one file for QGIS?

Thanks in advance for any help.

6

I wrote a guide posted here on the Landscape Laboratory blog (which requires use of the Mac command line):

The data comes as a zip file containing a folder for each major OS grid square, further broken down in many (2,800+) zip files each containing a single 10 x 10km .asc elevation file. For example:

terr50_gagg_gb.zip/data/tq/tq00_OST50GRID_20130401.zip/TQ00.asc

Unfortunately, even with a SSD drive, unzipping all the zip files, as recommended in the user guide, took around an hour and just under 1gb of disk space. Luckily, there is a way to keep these further zip files compressed after the first, main zip file has decompressed and create a single, lean file of the entire British Isles for quick and convenient use in your GIS using the Terminal and GDAL.

The elevation data is ultimately stored in .asc files. I made a list of all the .asc file names stored within the zip folders by decompressing all the individual zip files then running the following command from the /data/ folder with the Terminal (OS X in my case):

find . -name "*.asc" > ~/GB.txt

this outputted a text file listing each individual .asc file. Here’s an excerpt:

./data/tq/tq00_OST50GRID_20130401/TQ00.asc ./data/tq/tq01_OST50GRID_20130401/TQ01.asc ./data/tq/tq02_OST50GRID_20130401/TQ02.asc ./data/tq/tq03_OST50GRID_20130401/TQ03.asc ./data/tq/tq04_OST50GRID_20130401/TQ04.asc

To save yourself this time consuming stage, the text file I generated is available here (note to Ordnance Survey, it would be great if you provided something similar with future releases!). If you carried out the first step the unzipped folders can now be deleted. With a couple of find/replace commands within a text editor, the file paths were altered to

/vsizip//[path]/terr50_gagg_gb/data/tq/tq00_OST50GRID_20130401.zip/TQ00.asc /vsizip//[path]/terr50_gagg_gb/data/tq/tq01_OST50GRID_20130401.zip/TQ01.asc /vsizip//[path]/terr50_gagg_gb/data/tq/tq02_OST50GRID_20130401.zip/TQ02.asc /vsizip//[path]/terr50_gagg_gb/data/tq/tq03_OST50GRID_20130401.zip/TQ03.asc /vsizip//[path]/terr50_gagg_gb/data/tq/tq04_OST50GRID_20130401.zip/TQ04.asc

where ‘[path]’ is the file path to the terr50-gagg-gb folder. Note that ‘/vsizip//’ and the ‘.zip’ extension were included to allow GDAL to read the many individual .asc files whilst they remain zipped and combine them into a single ‘virtual raster’ mosaic. This edited .txt file can then be fed into GDAL build virtual raster program:

gdalbuildvrt -input_file_list ~/GB.txt ~/GB.vrt

After a very short time the resulting .vrt file is 1.3mb, and accesses the partially compressed terr50 folder that now only occupies 160mb or so disk space. Most importantly, the .vrt file loads into QGIS in a few seconds and can be used exactly like the .asc files would.

You will have installed GDAL alongside QGIS. Your copy of the GDAL build virtual raster will probably be located at /Library/Frameworks/GDAL.framework/Versions/1.11/Programs/gdalbuildvrt depending on the version number.

  • Thanks for your answer, it is much appreciated! I have created the text file but I am stuck at the next stage. I could not find 'gdalbuildvrt' anywhere in my computer so i redownloaded GDAL 1.11 complete with no luck. I then downloaded GDAL framework v1.11-0.4 and 'gdalbuiltvrt.cpp' has appeared in the downloads folder when i search for it. Is this the correct program? Also, how do I 'feed' the text file into this program to create the final merged image for qgis? – lilmawi Aug 10 '14 at 13:57
  • Did you download GDAL from here? The default install will put GDAL right where I listed. – Sorbus Aug 10 '14 at 15:22
  • I didn't realise there are two library folders. Ive found it now. Please could you explain to me how i feed the text file into this program? When i open this program it says: FAILURE: No output filename specified. logout [Process completed] at the bottom, is this ok? Thanks! – lilmawi Aug 10 '14 at 15:55
  • Okay, I'll assume your home folder is called 'will' and the file listing the tiles is called 'input.txt' and saved to the Desktop. You Open Terminal and type /Library/Frameworks/GDAL.framework/Versions/1.11/Programs/gdalbuildvrt -input_file_list /Users/will/Desktop/input.txt /Users/will/Desktop/output.vrt The output, 'output.vrt' will be created on the Desktop when the command is run. the -input_file_list bit means, I'm feeding you a file with multiple listed within it. – Sorbus Aug 10 '14 at 18:59
  • Thanks for explaining that to me, your answers are really helpful! I've sorted it now! In the latest edition of OS terrain 50, some of the file names are different, for example where some files used to be called 20130401.zip they are now called 20140717.zip so I had to create a new text document. – lilmawi Aug 11 '14 at 8:15
3

Sorbus has given an excellent answer (+1). The VRT solution in Sorbus' answer is definitely the way to go, and QGIS will also read the compressed vector contour files too (after the initial decompression). However, sometimes you may need access to the uncompressed data (e.g. if you are using something other than QGIS which doesn't support reading zipped gis files). So, just for the sake of completeness here is an approach to recursively unzipping the entire Terrain50 dataset in one go (which took considerably less than an hour - though I forget exactly how long).

This method requires 7Zip which is available for Windows, Linux, Unix and Mac. Unzip the downloaded file (it gives a folder containing subfolders named by maptile, which in turn contain various recursively zipped subfolders for both raster and vector contour data). Create a batch file (DOS example below - adapt to suit your OS) and save it in the topmost Terrain50 folder:

FOR /D /r %%F in ("*") DO (
    pushd %CD%
    cd %%F
        FOR %%X in (*.zip) DO (
            "C:\Program Files\7-zip\7z.exe" x "%%X"
        )
    popd
)

Obviously edit your path to 7Zip.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.