6

enter image description hereI am working with large tiled (no overlap) datasets of ortho imagery that has been mosaicked in ArcGIS and exported as a single, merged TIF for the entire AOI. These images need to be compressed so that they can be easily distributed to the public, however when compressing to JPEG straight from ArcGIS, some pretty horrendous compression artifacts appear in the imagery along the edge of the nodata boundary.

I am trying to find a way to eliminate these artifacts, while still maintaining a decent compression.

I have tried using GDAL nearblack followed by gdalwarp -srcnodata 0 -dstalpha input output and finally followed by gdal_translate input output -b 1 -b 2 -b 3 -mask 4 COMPRESS=JPEG --config GDAL_TIFF_INTERNAL_MASK YES

This has created a significant reduction in the black pixelated artifacts around the edge of the imagery - however these artifacts still do exist until I zoom into a scale of 1:1500 and then they disappear.

Any ideas of how to (if possible) eliminate these artifacts altogether?

Improve this question
7
  • The single, merged tiff from ArcGIS is obviously jpeg compressed because you need to use nearblack. Why don't you write them uncompressed or with lossless compression and continue from that with GDAL? Repairing artifacts afterwards will not make as good result.
    – user30184
    Sep 18, 2014 at 20:41
  • Unfortunately I have tried both - I merged the original TIFFs using GDAL with no compression and ran the same routine (minus the nearblack and another with the nearblack) and I still run into the same artifacts at the end. As mentioned, I need these to be compressed sufficiently for the public to be able to download at a reasonable rate from an FTP site.
    – Jesse
    Sep 18, 2014 at 20:58
  • Can you add some screen captures from the artifacts. Have you tried already different JPEG quality settings? You said that artifacts disturb if scale denominator is less that 1500. What is your pixel size? What compression rate is enough for you? My experience is that nobody complains if ratio with jpeg compression is 1:10 of even 1:20.
    – user30184
    Sep 18, 2014 at 21:10
  • I am just looking for a happy medium between data quality and size.
    – Jesse
    Sep 18, 2014 at 21:15
  • and the resolution of the imagery is .25cm
    – Jesse
    Sep 18, 2014 at 21:16

2 Answers 2

Reset to default
6

TL;DR The short version is to use a command like the following one which will make you as happy as it is possible with GDAL.

gdalwarp -of gtiff  -t_srs epsg:26912 -srcnodata 0 -dstnodata none -dstalpha -co compress=jpeg -co tiled=yes test.tif final.tif

Longer story

Your problem is not really in the compression artifacts but how to handle transparency with JPEG compressed TIFF files. It looks also that you have done right things and probably you have already read this thread from GDAL-devel mailing list http://osgeo-org.1560.x6.nabble.com/gdal-dev-Creating-jpeg-compressed-tiff-with-alpha-td4985300.html

I made a few tests with current GDAL development version 2.0-dev and it appears that the trick which are mentioned in the thread are mostly implemented into GDAL now in rather uses friendly way.

The following screenshots hopefully clarify what happens and what is doable with GDAL.

First image shows a detail from the border between data and nodata in warped and jpeg compressed image. I wouldn't say that there are any visible artifacts.

enter image description here

However, when setting the viewer to make black pixels transparent so that the blue background shows there are for sure nasty pixels. They appear because jpeg compression is lossy and some nodata pixels which were originally totally black (RGB 0 0 0) are only nearly black after compression, like RGB 0 0 1.

enter image description here

Compare the result with lossless LZW compression which keeps black as black. Compare also file sizes: LZW compressed 848 MB vs. 246 MB as JPEG compressed tiff. Note that the visual quality on the data area is about the same and the nasty things happen only at the border and if image is set to transparent.

The truth is that transparency can't be handled with transparent pixels with JPEG compression not other lossy compression methods. The mechanism that works is to attach to image an additional band that keeps the information about transparency. Transparency can be stored into an alpha channel or into auxiliary mask which can be even in a separate file. Both methods can be used with GDAL but unfortunately in your case neither of them lead to perfect result.

enter image description here

First shot is to convert nodata pixels into an alpha band with lossless method and compress the result into jpeg compressed tiff. The result is rather good but not perfect because as described in the mailing list thread, GDAL is compressing also the alpha channel with lossy jpeg method. However, while with nodata pixels the transparency is either zero of one but with alpha channel on range 0-255 the effect of lossy compression is not so bad.

enter image description here

The other method that GDAL supports is to write the data/nodata infromation into separate mask band. This info takes only 1 bit to store and it can be effectively compressed so the payload in disk space is minimal, in this case 1 MB extra for a 26855 by 23536 pixels sized warped image. The only drawback is that I do not know any viewer that supports the GDAL external mask layers. Not ever QGIS which is heavily based on GDAL with image handling understands them. There seems to a related ticket but that does not mention external masks http://hub.qgis.org/issues/3348.

Improve this answer
4
  • Thanks, I will play around with this a little more and see if I can get something to work. Unfortunately, I am receiving an error when trying to JPEG compress a 4 band image with the argument outlined above. I appreciate the background information - hopefully I can find some happy medium.
    – Jesse
    Sep 19, 2014 at 18:50
  • 1
    +1 It is nice to see a careful illustrated analysis of an issue: it is superior to and more valuable than a mere "use this code" solution. Whether or not readers use GDAL or QGIS, we all benefit from understanding the issue and from the several options discussed here.
    – whuber
    Sep 19, 2014 at 19:06
  • Note that ArcGIS 10+ supports GDAL mask bands
    – user2856
    Sep 20, 2014 at 17:01
  • Happily ArcMap, Pro and Qgis all support mask bands out fo the box now. See gis.stackexchange.com/questions/387877/… for way to add them with nearblack.
    – matt wilkie
    Mar 19, 2021 at 21:00
0

As you probably know, jpg is a lossy compression, so you cannot completely avoid artefacts. I suggest that you export in TIFF with COMPRES=LZW instead, which will preserve your data.

Improve this answer
1
  • I will try with the LZW compression and see how large the files end up. Thanks.
    – Jesse
    Sep 18, 2014 at 20:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.