completing satellite imagery tiles when data coverage is less than 100%

Question

I want to join several images (>=2) into one "best" image. Best is defined on low cloud cover and high data coverage. An example using free Sentinel satelite data follows.

See http://sentinel-s2-l1c.s3.amazonaws.com/tiles/12/S/XB/2017/6/1/0/preview.jpg and http://sentinel-s2-l1c.s3.amazonaws.com/tiles/12/S/XB/2017/6/ for the source of images below.

Are there any algorithms or processes for completing satellite imagery tiles that don't have 100% data coverage in order to generate a full tile?

See visualizations below for examples of what I mean.

I'm not too familiar with the literature, and don't know what the terminology that I should be looking out for is.

Example:

Example:

Answer 1

For images of the same location but different dates, I would rather talk about compositing than mosaicing (which combines images from different extents into a larger image). You will find a lot of details if you search "compositing" keyword, but here is a short summary:

There are two main approaches for the compositing of time series:

• Best available pixel approach (select the "best" pixel at each location based on a given criteria, e.g. use the pixel with the maximum NDVI value or closest non cloud pixel to the central date of the compositing period). An example with Landsat can be found here

• Combined pixel approach (e.g. take the average of all pixels at the same location (mean compositing) or use a temporal regression to interpolate "missing" pixels at some dates (gap filling) ). Note that gap filling potentially creates one image at any date (and you decide the one you keep), while compositing gives only one image per compositing period (you can use a sliding temporal window, but it is less "precise" temporally).

"Mean compositing" has been used in several successfull projects with MERIS and SPOT VGT (see here). "Max NDVI" compositing is used for the MODIS composite. Interpolation at some dates of interest has been done here with Sentinel-2 images. Personnally, I prefer the "combined pixel" type approach.

Now you must be aware that the quality of your compositing depends a lot on the quality of your inputs, especially if you don't have a large number of input data (sentinel-2 is "only" every 5 days, not every day like Sentinel-3) :

• good cloud mask (including cloud detection, haze detection, cirrus (high altitude thin cloud) detection and cloud shadow detection.

• top of canopy reflectance : convert Digital Numbers from the satellite into meaningful reflectance values, including corrections from BRDF (the light is not homogeneously reflected in all directions and there is an impact of the surface on the differences), atmospheric correction and topographic correction.

• good registration between the different images. pixels must represent the same location as much as possible.

• sometimes also: temporary event detection (floods and snow)

Note that a software has been developed in the frame of an ESA project (SEN2AGRI) for creating cloud free composites.

Bonus: examples of global composites

• with Sentinel-2 (WMS only)

• with MERIS, SPOT VGT and AVHRR surface reflectance (can be downloaded)

• with Landsat circa 2000 and circa 2014 (median value using full Landsat archive)

Answer 2

I think that what you describe is still part of what is called mosaicing (or image stitching). Mosaicing involves indeed joining adjacent tiles, but usually tiles do have some overlap.

Here you are interested in particular in two steps:

1. Stitching the images: i.e. finding the correct overlaping position

2. Blending the overlapping pixels

There is an excellent survey of the different methods for each step in this paper: Ghosh and Kaabouch (2016) A survey on image mosaicing techniques, J. Vis. Commun. Image R. 34 (2016) 1–11