3

I would like to use gdalwarp and gdal_translate with some very large raster images (313,040 x 313,040 px) and I'm wondering what the best format or method is to do this, if it's even possible.

I have the file in Photoshop, which at that size can only output Large Document Format (.psb) files, which I don't think gdal can read.

Will gdal even be able to handle such a sized document?

If so, how can I convert it to a format gdal can use?

  • do you have the files in other format? (tiff?) how many ram do you have? what other software do you manage? – Pau Aug 18 '14 at 16:08
  • Beyond the issue of creating this huge image, I wonder about the practicality of actually analyzing such a massive dataset. Many (most) spatial analysis tool that you may plan to use (if any) will likely require reading in the whole dataset or large chunks of the data into memory. Unless you have a massive amount of RAM, this will be impossible and you are setting yourself up for analysis errors a plenty. – WhiteboxDev Aug 18 '14 at 16:45
  • @WhiteboxDev OTB works fine with large images, and it is available from QGIS. On the other hand, .vrt are great alternatives to building huge files. – radouxju Aug 19 '14 at 7:25
3

with gdal, very large files can be handled in the bigtif format, an extension of the tiff library. I recommend that you use the -co TILED=YES option when working with very large files, and you can force bigtif using -co BIGTIFF=YES option (the output extension remains .tif).

  • and -co COMPRESS=LZW – mdsumner Aug 18 '14 at 20:33
  • You should not have any troubles with gdal_translate and tiled bigtiff. If you want to spend money to licenses then ECW or JPEG2000 with ECWJPEG2000, Kakadu of MrSIDJP2 drivers will also work fine. But warping big images is not as painless. You may need to play with config options trac.osgeo.org/gdal/wiki/ConfigOptions for getting acceptable speed and it is better to warp into uncompressed format and compress the result later. I would myself clip the output into rectangular pieces of about 50000 by 50000 pixels with gdalwarp -te and mosaic and compress the tiles with a second run. – user30184 Aug 19 '14 at 6:59
1

I'm a little late to the party, but this may help others.

Photoshop has the ability to read/write 'raw' format files. On opening the file you have to tell it how large the header is, if there is one, plus the dimensions, number of bands, and band layout (interleaved or no). GDAL can read/write the ENVI file format, which is essentially a raw file with a separate header file.

In a nutshell, get your image in ENVI format, rename the .env file to .raw, then open it in Photoshop. You can get the necessary info from the header file (.hdr) or the output from gdalinfo or the likes. Make your changes in Photoshop and save as a new .raw file, making sure to double-check the settings (band interleaving may get turned on by default, I can't remember). Rename the .raw to .env and do your processing with GDAL.

If you become familiar with the ENVI header files, you can manually create one for any raw file you save from Photoshop; you just have to fill in the proper details.

I do this all the time for very large (~80gb) geospatial image files. Heck, I do it for small files too, because it's a very easy way to keep your geo-information intact.

0

This is an other format, that can handle very large Raster Files: PCI Geomatics. It can be the direct output of gdalwarp, which is not the case for many Formats.

A further alternative is the usage of the WarpEcwTool. This tool was specially written to warp huge ECW files. The tool cuts the input file into smaller tiles, warps them one by one an merges the warped tiles in the last step.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.