1

Background

According to AWS's Copernicus Digital Elevation Model page,

The Copernicus DEM is a Digital Surface Model (DSM) which represents the surface of the Earth including buildings, infrastructure and vegetation. We provide two instances of Copernicus DEM named GLO-30 Public and GLO-90. [...] Data is provided as Cloud Optimized GeoTIFFs.


Aim

I would like to access these GeoTIFFs, using Python. I would like to be able to specify a bounding box, and get those GeoTIFFs which overlap or intersect with that bounding box.


My attempts

1) Access AWS directly

After installing boto3 (with pip3 install boto3), I do, relying on this answer to the question Can I use boto3 anonymously?:

import boto3
from botocore import UNSIGNED
from botocore.client import Config

s3 = boto3.client('s3', region_name='eu-central-1', config=Config(signature_version=UNSIGNED))

Then I query for list of objects in the bucket, using the second line of this answer to the question Use boto3 to download from public bucket:

objects = s3.list_objects(Bucket='copernicus-dem-30m')

I get a dict, with keys:

objects.keys()
# outputs: dict_keys(['ResponseMetadata', 'IsTruncated', 'Marker', 'Contents', 'Name', 'Prefix', 'MaxKeys', 'EncodingType'])

I access Contents, which is a list, so viewing the first element:

objects['Contents'][0]

I get:

{'ETag': '"e0c8b05f7999eedd83ba85e5c94cb670"',
 'Key': 'Copernicus_DSM_COG_10_N00_00_E006_00_DEM/Copernicus_DSM_COG_10_N00_00_E006_00_DEM.tif',
 'LastModified': datetime.datetime(2020, 11, 25, 7, 51, 3, tzinfo=tzlocal()),
 'Size': 3706379,
 'StorageClass': 'STANDARD'}

Viewing objects['Contents'][0]['Key'] gives me Copernicus_DSM_COG_10_N00_00_E006_00_DEM/Copernicus_DSM_COG_10_N00_00_E006_00_DEM.tif, which is a string. So this didn't got me much closer to accessing the actual TIFF data in a Python notebook.


2) Using opentopography.org

Googling "search in GLO-30 bucket" lead me to OpenTopography's search map. After selecting a small region on the map & filling other other details (like email), and waiting a few moments, I get to:

enter image description here

where I can download a .tar.gz file. Upon extraction, I find a .tif file.

I believe this is an example of the files I am looking for, but this method does not use Pyhton at all, and it is not suitable for automated processes.


Question

How do I access the TIF files using Python?

I am looking for a similar thing to this this:

geotiffs = client.search_geotiffs(bounding_box=[59,10,60,11])

where the returned geotiffs is a list of tiff files from the GLO-30, intersecting or overlapping with the bounding box defined by points 59N10E & 60N11E, for example.

1

From the boto3 documentation, here's how you download a file:

s3 = boto3.client('s3')
s3.download_file('BUCKET_NAME', 'OBJECT_NAME', 'FILE_NAME')

In your example it might look like this

key = objects['Contents'][0]['Key']
s3.download_file('copernicus-dem-30m', key, key.split('/')[-1])

As for downloading all overlapping tiles, the numbers in the filename reference the lower left (south west) corner of the tile. The code to generate the filenames will look something like this:

  • Generate all northings as an inclusive range between math.floor(bounding_box_south) and math.ceil(bounding_box_north) - 1). Do the same for eastings.
  • Take every combination of northing and easting, convert into N00_00_E006_00 format. Copernicus doesn't have fractional tiles, it's all [N|S]xx_00_[E|W]xxx_00.
  • Format that into an s3 key like the one in your question.
  • Download with boto3.
2
  • Very cool. Thanks. If I don't straight want to download, but I'd like to get an URL I can use to download the tiff (by pasting it to my browser, for example), do you know how to modify this script? I guess I should switch the last line to something, I don't see to what yet.
    – zabop
    Sep 29 at 11:59
  • Posted a question here about the above issue: stackoverflow.com/questions/69376206/…
    – zabop
    Sep 29 at 12:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.