I have ICON datasets (.grib2.bz2 file format) from Deutscher Wetterdienst (DWD) in triangular grid that I would like to transform to a regular grid. Currently, I run the following statement in my Python script:

subprocess.call(['./transform.sh', blobName])

In this bash script I have the following:

bzip2 -d ${1}

export in_file=${1//.bz2/}
export out_file=${in_file//icosahedral_single-level/world_0125x0125}

time cdo -f grb2 remap,settings.txt,weights.nc ${in_file} ${out_file}

DWD provides details on how to achieve this here. I would like to improve my script in two ways:

  1. Run everything in python itself
  2. Do not use physical files

This means I would like to download the .grib2.bz2 file in memory and produce the output in memory so I can store it in Azure Blob Storage.

Is this feasible? Maybe with rasterio? For 1) I could use python cdo but this does not solve 2) as far as I know. I was also not able to run cdo on Python 3.X version.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.