While I don't know why GDAL provides this overlap in functionality, be sure to set the cache for gdalwarp to make it really fast:
# assuming 3G of cache here:
gdalwarp --config GDAL_CACHEMAX 3000 -wm 3000 $(list_of_tiffs) merged.tiff
Be sure to not define more cache than having RAM on the machine.
I've found when something isn't particularly well documented in GDAL, that looking through their tests can be useful.
The /vsis3 test module has some simple examples, though it doesn't have any examples of actually reading chunks.
I've cobbled together the code below based on the test module, but I'm unable to test as GDAL /vsis3 requires credentials ...
I just happened to come across this question and a potential answer when looking for something else.
gdal_merge.py uses nearest neighbor resampling. If you want control
over the resampling used, you should use gdalwarp instead.
Add an empty argument in the first approach (because gdal_merge.py parses arguments starting from 1 and not 0):
import gdal_merge as gm
gm.main(['', '-o', 'merged.tif', 'N00W078.tif', 'N00W079.tif'])
Join the path of gdal_merge.py in the second approach:
import os, subprocess
gm = os.path.join('C:\\','...
Since /vsis3/ is implemented in GDAL you can also use rasterio to read Windows of S3 datasets. This requires either your credentials to be set up for boto or using rasterios AWS session handler.
with rasterio.open('s3://landsat-pds/L8/139/045/LC81390452014295LGN00/LC81390452014295LGN00_B1.TIF') as ds:
window = ds.read(window=((0, 100), (...
This was a tough problem to track down because I had thought that the effect was at the edges of tiles when it fact they are throughout the data. You're right that the phenomena isn't in the data before mosaicking the data. The problem results from the resampling process that is inherent in mosaicking. You need to use either the cubic convolution or bilinear ...
You could build a virtual raster which allows you to merge multiband rasters. This should be less memory-intensive than the gdal_merge tool. You can access this from the same menu:
Raster > Miscellaneous > Build Virtual Raster (Catalog)
The easiest way to do this is by importing the path where gdal_merge.py is located, in my case, /usr/bin/ -- substitute with the path to gdal_merge on your system, which, obviously, could be a Windows path too.
import gdal_merge as gm
You will now have to build up an array for sys.argv, as if you were calling ...
I also needed to know how to rename bands in an open source environment so spent a whole day looking for answers.
There is no way to name bands in QGIS while merging.
But it can be done after the file is created, by editing their .aux.xml file. It works for both .tif and .img files, as far as I've researched.
The solution is to include after each <...
gdal_merge.py is the correct tool to 'stack' your input images.
Assuming that your first band has a valid color table you could use:
gdal_merge.py -separate -pct -o output_file.tif file1.tif file2.tif file3.tif
Note: The command has been reformatted with -o output_file.tif before the list of inputs.
From the docs:
Grab a pseudocolor table from ...
This can be achieved with the help of GDAL's Virtual Raster Format. With this you can essentially skip the step of creating one giant DEM. The VRT will be handled by GDAL like a giant, merged DEM but is just a small XML file containing the file paths for each tile as well as some metadata. This can then be fed to gdalwarp together with a bounding box or a ...
I ran across this mosaicing the True Marble imagery as well, though I used gdalbuildvrt and then gdal_translate. From memory, the recalcitrant tiffs are stored as a single band with a color table.
Just convert them to 3 band RGB with gdal_translate:
gdal_translate -expand rgb TrueMarble.250m.21600x21600.B4.tif TrueMarble.250m.21600x21600.B4.RGB.tif
Open MSYS (It should have been downloaded along if you used OSGeo4W utility to install qgis)
cd (Change Directory) to your folder with your data.
cd c:/(path to)/(my data)/ (Hint: pressing tab, autocompletes)
one-line it: gdal_merge.py -o out.tif $(ls *.tif)
out.tif -> is your output file, name it to whatever you want.
$(ls *.tif) -> lists all the files ...
You can't tell gdal_merge.py to process a specific band from multiband rasters as it doesn't support a -b band argument or path/to/file.tif/band_num syntax.
I would use gdalbuildvrt then gdal_translate:
gdalbuildvrt -b 1 -input_file_list "file_list.txt" "target.vrt"
gdal_translate "target.vrt" "target.tif"
Or without an intermediate VRT file:
No, if your bands all have the same resolution, no resampling will occur when using gdal_merge.py. So it's perfectly fine to use it.
As for "Pansharpening": Since the process aims to turn a low-res color image into a high-res color image with the help of a high-res panchromatic image, naturally resampling is involved. In a common implementation this would ...
It is nearest neighbor as written in http://lists.osgeo.org/pipermail/gdal-dev/2006-November/010619.html
There is also a hint in the mail "If you
want control over the resampling used, you should use gdalwarp instead."
gdal_merge.py -o output.tif `ls *.tif`
The back ticks mean execute whatever is inside the back ticks before the main command, so this will find all tif files in current directory, which will then be used as the input to gdal_merge.py.
Instead of backticks, you can also use the $(command) syntax, ie,
gdal_merge.py -o output.tif $(ls *.tif)
is equivalent ...
Did you do ldconfig to be sure that your changes to LD_LIBRARY_PATH are really applied?
Where is your libgdal.so.20 file?
Did you look at the GDAL .travis.yml file? A Travis file instructs how you can compile, deploy code for testing. It's firstly for testing code purpose but you can also use it to find out if you didn't miss a point when compiling.
gdal_merge allocates memory for whole raster at once so it runs quickly for datasets that fit into memory. If it is not you case, use gdalwarp tool which does tiling so you can control how much memory does it use:
gdalwarp --config GDAL_CACHEMAX 512 -wm 4096 merged.tif
where GDAL_CACHEMAX is memory for IO cache and -wm is memory limit which controls the ...
You should read again the document page or gdalbuildvrt http://www.gdal.org/gdalbuildvrt.html. Pay attention to parameter called "resolution"
In case the resolution of all input files is not the same, the -resolution flag enables the user to control the way the output resolution is computed. 'average' is the ...
To put the comments into an answer:
It looks like the processing was doing a bad job at predicting the optimal compression? Coupled with no tiling there was really inefficient storage.
The solution was to use the geotiff creation options PREDICTOR=2 & TILED=YES.
The final command to call is then
gdal_merge.py *.tif -o ~/out.tif \
-co COMPRESS=LZW -...
The chosen value is stored in the PhotometricInterpretation TIFF tag, which defines the color space, i.e. how to interpret the values of the bands. Example for a 3-band GeoTIFF defined as RGB: band 1 = red, band 2 = green, band 3 = blue. CMYK stands for the Cyan, Magenta, Yellow, blacK color space, etc.
Answering my own question: Those black pixels in the images represent a "NoData" value. If I tell the gdal merge code that pixels with a color value of 0 are "NoData," it can correctly stitch the images together. (The "NoData" value isn't necessarily set to 0, it can be any value not otherwise used in the image, but in the case of NAIP ortho images, it was ...
If you are in a windows environment you can create a batch file:
for /f %%A in ('dir "c:\your_Path\*.tif" /b/s') do (
"c:\path\to\GDAL\bin\gdaladddo" -ro %%A 2 4 8 16 32
This is a batch file that says "for this folder and every subfolder, every file with the '.tif' extension run gdaladdo". You will need to change c:\path\to\GDAL\bin to ...