4

I want to shift from downloading Landsat8 imagery directly from USGS to the AWS service (https://pages.awscloud.com/public-data-sets-landsat.html).

I have some automated processes that are not accepting the TIF files from AWS. Upon closer inspection the file sizes differ substantially.

For example for scene LC80930842016347LGN00 if I download from EarthExplorer as LC80930842016347LGN00.tar.gz and extract all bands except B8 are 116,191KB.

When I download the bands individually from AWS (http://landsat-pds.s3.amazonaws.com/L8/093/084/LC80930842016347LGN00/index.html) all bands except B8 are around 60MB and the BQA band is only about 1MB.

Is there some compression going on in the AWS storage and if so, is there a process to transform these back to original so the Landsat_LDOPE tools will work?

  • 2
    gdalinfo tells me the AWS tifs are DEFALTE compressed with 512x512 tiling. – user2856 Dec 16 '16 at 4:29
  • According to the TIFF wiki article DEFLATE compression is lossless so we in theory should be able to convert back to original state? – Ben Dec 16 '16 at 5:53
  • So the answer to the question in the title is probably "because as DEFLATE compressed they take less disk space without loosing quality". – user30184 Dec 19 '16 at 8:00
1

AWS compresses GeoTiff using DEFLATE.

Running

gdal_translate LC80930842016347LGN00_B1.TIF LC80930842016347LGN00_B1_.TIF

returns uncompressed file exactly the same size as TIFs from USGS.

  • It may not be the compression, but the tiling that ldope chokes on. I suggest testing compressed without tiling as well. – user2856 Dec 16 '16 at 7:14
  • Cheers, I'll try. Actually the ldope is still choking on my above solution, but Spectral Transformer is working again. – Ben Dec 16 '16 at 7:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.