Trying to run GDALbuildVRT in command line

Question

I have a directory of OS data stored as tiff files (approximately 2000), which I want to create a virtual raster with. I understand that you can run this in the QGIS UI but because of the large number of files it is not possible (see: QGIS virtual raster - insufficient permissions)

I therefore want to run this in the command line of which I have very little experience. My workflow so far is this:

1. I have changed the directory to the "data" folder where all of the raster tiles are saved.

2. The OS tiles are saved in here within there own BNG folders

3. I then use the following code to try and build a VRT with all the tiff files that are within that directory with a CRS of BNG 22770.

gdalbuildvrt Vector_district.vrt data/*tif

and get this message:

Can someone please help me with this bearing in mind I have little to no experience in coding or the command line?

Answer 1

The problem is that this construction is trying to look for a "data" folder within your existent "...\Vector Map District Raster\data" folder. Since there is no such "...\data\data" folder, it returns an error.

You'd need to iterate through each folder, but as far as I know there is no way to do this with gdalbuildvrt (or any other GDAL utility, for that matter). You have three options here, as I see it:

1) Put all your TIFFs in a single folder. This way, you can call the utility as:

gdalbuildvrt Vector_district.vrt "folderName\*.tif"

2) Create a list of file paths for each raster, and put it in a text file (one filepath per line), as such:

"\HP\raster1.tif"
"\HP\raster2.tif"
"\HT\raster1.tif"
"\HU\raster1.tif"

and so forth. Save this list as a txt, and call it in the utility:

gdalbuildvrt Vector_district.vrt -input_file_list myRasterList.txt

3) Do it in python. For this you'll need the GDAL python bindings, which you can get from here. Then you do:

import os
from osgeo import gdal

os.chdir(r'C:\Users\*****\Desktop\Os Raster\Vector Map District Raster\data')
li_dirs = [folder for folder in os.listdir(os.getcwd()) if os.path.isdir(os.path.join(os.getcwd(), folder))]

li_all_files = list()
for folder in li_dirs:
    path = os.path.join(os.getcwd(), folder)
    li_files = [file for file in os.listdir(path) if os.path.splitext(file)[-1] == '.tif']
    for file in li_files:
        li_all_files.append(os.path.join(path, file))

gdal.BuildVRT('Vector_district.vrt', li_all_files)

Answer 2

*tif will not get expanded into subdirectories of .../data. You must use an input file list as described in http://www.gdal.org/gdalbuildvrt.html

-input_file_list:

     To specify a text file with an input filename on each line

You can create such file from command line when you home directory is at you "data"

dir /b /s *.tif >my_input_file_list.txt

Answer 3

The solutions you gave above are great. I am adding my piece of the answer:

I create two .Bat files to create mosaics of TIFF files by subfolder

First I create a .Bat file to loop in subdirectories and get the TIFF list from each one. Considering I have two different names of images, I use two wildcards

@echo off
set back=%cd%
for /d %%i in (C:\GeoData\Your_Main_Directory\*) do (
cd "%%i"
dir /s/b *MS.tif *RS.tif > list.txt
)
cd %back%

Then, another .Bat to loop in the same subdirectories (and those inside) and do the VRT mosaic.

@echo off
set back=%cd%
for /d %%i in (D:\GeoData\Planet\*) do (
cd "%%i"
gdalbuildvrt -resolution highest -srcnodata 0 -input_file_list list.txt Mosaic_VRT.vrt 
)
cd %back%