6

When using Python API for GEE, is it possible to request values from a server asynchronously?

For example, in JS API one can use .evaluate(), but in Python it seems like there's only .getInfo() request, which temporary blocks the following execution of the script. Does it mean that it is not possible to make async callbacks with Python API?

1
  • 1
    I haven't found a way to do this, and rewrote a ton of code to JavaScript because of it. I very much would like to see a solution to this too. Commented Jun 3, 2021 at 18:40

2 Answers 2

4

Yes, it is possible using one of the few available libraries with similar functionality.

Asyncio is the first thing that comes to my mind https://docs.python.org/3/library/asyncio.html You basically get the functionality of async/await, it's possible to create callbacks.

Here is a step by step comparison of pulling data asynchronously using asyncio and the same functionality in node js: https://medium.com/@interfacer/intro-to-async-concurrency-in-python-and-node-js-69315b1e3e36 If you're familiar with Javascript it should be easier to understand.

If you need a more sophisticated solution, especially for running relatively heavy tasks in the background I'd recommend another of the well known Python modules - Celery https://docs.celeryproject.org/ which allows to execute functions in isolated (separated from the app that triggers the job) workers. The basic idea of Celery is that functions you want to run asynchronously, must be wrapped with Celery that is listening to a message broker (Rabbit MQ, Redis, SQS etc.), then invoking that function from the main app using dedicated methods sends arguments to the message broker and they are passed to the worker which executes the function and notifies the message broker that job has been finished. It may sound a bit complicated but I recommend to read the docs and check examples.

3
+50

This would be my simple solution using the threading module. I think asyncio is too much for a simple use case, but could be implemented in a complex situation like a web app.

import ee
import threading
ee.Initialize()


def retrieve(f):
    def wrap(obj, *args):
        f(obj.getInfo(), *args)
    return wrap


def evaluate(obj, callback, args):
    args.insert(0, obj)
    callback = retrieve(callback)
    thd = threading.Thread(target=callback, args=args)
    thd.start()
    

# EE object
eeobj = ee.List([1,2,3,4,5,6])
    
# use case 1: printing values
def callback1(obj, n):
    print([i*n for i in obj])

evaluate(eeobj, callback1, [5])

# use case 2: filling a container
container = []
def callback2(obj, cont):
    """ filling a container """
    for item in obj:
        cont.append(item)
        
evaluate(eeobj, callback2, [container])
# wait 2 sec and...
print(container)

For testing it (in Spyder for example) I recommend to run it by parts. If you run all at once you won't be able to see the behavior properly.

UPDATE

you can use it through the geetools package

import ee
ee.Initialize()
import geetools

# EE object
eeobj = ee.List([1,2,3,4,5,6])

# use case 1: printing values
def callback1(obj, n):
    print([i*n for i in obj])

geetools.evaluate(eeobj, callback1, [5])
1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.