Exporting ERA5 Image Collection data from Google Earth Engine

Question

I'm trying to export the ERA5daily Image Collection data from Google Earth Engine using the GEE API in python. I need the Max Temp, Min Temp and Average Temp for the time between 1979 and 2016 (in the example here just 1 day for convenience), within a specific bbox.

Basically I need the temperature time series (max, min, avg) for each grid within my geometry object, because I want to calculate different indices of temperature variability and extremes afterwards.

Unfortunately I haven't had much success so far. I managed to export single .tif images with the GEE Code Editor in JavaScript but it seems quite unreliable and I would have to press 'Run' thousands of times. I've read that for batch exports the Python API is recommended.

Below is the code I have tried so far, adopted from GEE Tools

This code works for exporting the images in .tif, however I'm running into several problems. First I don't manage to select the bands. In the GEE code editor this works with

var col = ee.ImageCollection('ECMWF/ERA5/DAILY').select()

But in Python it just doesn't export anything to drive when I use it.

Secondly I can't figure out what the scale parameter does. The default setting is 30, which leads to massive files (and the need to increase maxPixels). When I set it to a higher value however the data stored in the .tif changes. So not really sure what to do about that.

I know there is likely a much better way to extract such large time series to reduce file size.

from geetools import batch
import ee
ee.Initialize()

region = ee.Geometry.Rectangle([79, 25, 89, 31])

col = ee.ImageCollection("ECMWF/ERA5/DAILY")\
    .filterDate('1979-01-02', '1979-01-03')\
    .filterBounds(region)

batch.Export.imagecollection.toDrive(col, "ERA5", region=region)

Answer 1

you will find documentation on this page : https://developers.google.com/earth-engine/python_install

I propose you to go through the native export tool of Earth Engine instead of geetools.

 import ee
 from google.colab import drive

Authenticate and mount your drive into your project

 ee.Authenticate()
 ee.Initialize() 
 drive.mount('/content/drive')

You can export images using a thread. I've never tried to export a whole collection but it seems to me that you can do it by replacing the 'image=image' attribute by 'collection=yourCollection' (not sure of that). In general, I prefer to transform my collection into an images list and then to upload images one by one.

imagesList = collection.toList(collection.size())
for i in range(0,imagesList.size().getInfo()):
  image= ee.Image(imagesList.get(i))
  #Process the export for you image into the folder of your choice into Drive 
  task = ee.batch.Export.image.toDrive(image= image,
    description='Exported from EarthEngine',
    fileNamePrefix='filename.tif',
    scale= scale,
    folder='repertoryOfYourChoice/',
    fileFormat='GeoTIFF')
 task.start()

Then you can follow the status of the last uploaded image in your list (you can upgrade this code by storing all the tasks in a list to track the status of all the images in your list) :

print(task.status())

Otherwise, about the scale it allows you to reproject an image to have a different resolution and speed up the export time. So it can be useful to use a large scale when working on very large areas. However, be careful not to scale it too large because the calculation time of the reprojection could sometimes be too greedy and exceed the maximum memory size allowed.

Answer 2

Two things: Firstly you can get the native scale of an ImageCollection at the equator using scale = collection.first().projection().nominalScale();

However since what you are saying is

Basically I need the temperature time series (max, min, avg) for each grid within my geometry object, because I want to calculate different indices of temperature variability and extremes afterwards.

If I was you I would really think about the need to download the ImageCollection or if you could do with staying in the Earth Engine Environment. It seems like you are quite new to Earth Engine, so let's say you just want to see if there's a positive trend in the average temperature for your study site. In Earth Engine you would accomplish it something like this (in the Earth Engine Javascript Code Editor):

    // This function adds a band representing the image timestamp.
var addTime = function(image) {
  return image.addBands(image.metadata('system:time_start'));
};


var trend = ee.ImageCollection("ECMWF/ERA5/DAILY")
    .filterDate('1979-01-02', '1979-12-02')
    .filterBounds(region)
    .select("mean_2m_air_temperature")
    .map(addTime)
    .reduce(ee.Reducer.linearFit())

Map.addLayer(trend.select("scale"))

You can find more info on linear regression in Earth Engine here: Earth Engine Linear Regression

If you want to do other things than a linear regression there are many more Reducers available, which can calculate all sorts of things for you.