Context: I process a set of image using the Google Earth Engine Python API. I want to apply the process on a set of image and save the output (in my case export to GEE Assets).

What I have done: Looping through the image set and export each image one by one. That works well.

Problem: since I would like to apply my process over 60k images, I face the number of task restriction (3000).

Traceback (most recent call last):

ee.ee_exception.EEException: Too many tasks already in the queue (3000). Please wait for some of them to complete.

What I found: the geetools modules. However, by looking at the code, it also seems to iterate over the image collection and exports each image one by one.

Question : Is it possible to export multiple images in one task ? What are the solutions ?

Here the code:

col = ee.ImageCollection("COPERNICUS/S2") \
        .filterDate("2018-01-01", "2018-12-31") \

# Get names of all image as a python list
image_names = get_name_collection(col).getInfo()
total = len(image_names)

# Create new image collection if don't existe
folder = "users/ab43536/masks_4_methods"
if folder not in [elt["id"] for elt in ee.data.getList({"id": "users/ab43536"})]:
    ee.data.createAsset({'type': "ImageCollection"}, folder)

# Apply process to all images
for i, name in enumerate(image_names):
    print("{:4d}/{} = {:05.2f}%   Image {}".format(i, total, i / total * 100, name))
    # The long process
    mask = computeCloudMasking(name)

    # Export image one by one
    export_image(mask, folder, getGeometryImage(ee.Image(name)),
                 name.split('/')[-1], num=i, total=total)

Where export_image is:

def export_image(image, asset_id="users/ab43536/", roi=None, name=None, num=None, total=None):
    """ Export one image to GEE Asset
        :param image: image to export
        :param roi=None:  specify the roi, default compute from image dimension
        :param name=None: name of the image
    if roi == None: roi = getGeometryImage(image)
    if name == None: name = image.id().getInfo()
    description = "Default export"
    if num != None and total != None:
        description = "Image {} on {} equal {:05.2f} pourcent".format(num, total, num / total * 100)
    # print(description)
    assetId = asset_id + name

    # Create a task : export the result as image asset
    task = ee.batch.Export.image.toAsset(image=image.clip(roi),
  • 1
    Hi Alexander, the developer of geetools here. You can ask your question here if you want. It has a way to export an ImageCollection as an ImageCollection asset Commented Jul 5, 2019 at 11:03
  • Hi @RodrigoE.Principe. Thank you for reply. Thanks to your work, I founded how to create an ImageCollection using: (ee.data.createAsset({'type':"ImageCollection"}, "users/.../..."). Commented Jul 5, 2019 at 14:36
  • 1
    You're right, geetools is not handling this in a proper way, I'll fix that! Thanks! Commented Jul 5, 2019 at 15:29
  • If you want it to get out of on hold you have to include some code. What you have tried, what didn't work, errors.. Commented Jul 6, 2019 at 12:39
  • @RodrigoE.Principe Why using your code I get ee.batch.Export.image.toAsset(...) => "can't overwrite"?
    – Giox
    Commented Oct 4, 2019 at 12:24

1 Answer 1


Processing that amount of images in a stack would be too big, but you can export multiple images in one task by stacking the images, then exporting the stack.

Here's my code for the Java version, if you know how to translate:

// Stack collection to output image

var tsStack = function(image, list) {
    return ee.Image(list).addBands(image.select([bands]));
var stack = ee.Image(newcol.iterate(tsStack,ee.Image(0.0).toDouble())).slice(1)

Export.image.toDrive({image:stack,description:name+'_stack',folder:'yourfolder', region:roi, scale:scale, crs:crs})

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.