1

A lot of remote sensing imagery from the USGS and NASA is hosted behind an Earthdata login account (which is free). I'd like to be able to open the files directly using rasterio and passing along my credentials. I can do this with fsspec. But I'd like to use only rasterio with the built in gdal authentication option, which is not working.

So the following works:

import rasterio, fsspec, aiohttp

username, password = 'earthdata_login', 'earthdata_pw!'

tif_file = 'https://data.lpdaac.earthdatacloud.nasa.gov/lp-prod-protected/HLSS30.020/HLS.S30.T55JCN.2015332T001732.v2.0/HLS.S30.T55JCN.2015332T001732.v2.0.B07.tif'

with fsspec.open(tif_file, client_kwargs={'auth': aiohttp.BasicAuth(username, password)}) as fobj:
    with rasterio.open(fobj) as src:
        print(src.profile)

But the following gives an IO error.

with rasterio.Env(GDAL_HTTP_AUTH = 'BASIC', GDAL_HTTP_USERPWD = f'{username}:{password}'):
    with rasterio.open(tif_file) as src:
        print(src.profile)

How can I open files with rasterio using the GDAL authentication option?

1 Answer 1

2

Two ways I can think of:

  1. Create a token
  2. Use a .netrc file

Both of which I prefer to keeping secrets in the code (e.g. username, password = 'earthdata_login', 'earthdata_pw!' ) as there's much less chance of accidentally committing them to a public repository like GitHub.

For 1. hdr.txt contains:

Authorization: Bearer edfkjdfsdlkfSdlkfjSLDkfsldkf?SLDKfm?SDKfm?SLDKfmetc...etc...

Code:

with rio.Env(GDAL_HTTP_HEADER_FILE="hdr.txt"):
    with rio.open(tif_file) as src:
        print(src.profile)

Output:

{'driver': 'GTiff', 'dtype': 'int16', 'nodata': -9999.0, 'width': 3660, 'height': 3660, 'count': 1, 'crs': CRS.from_epsg(32655), 'transform': Affine(30.0, 0.0, 300000.0,
       0.0, -30.0, -2700000.0), 'blockxsize': 256, 'blockysize': 256, 'tiled': True, 'compress': 'deflate', 'interleave': 'band'}

For 2. .netrc contains:

machine urs.earthdata.nasa.gov login some_user password ABCdef123!

Code:

with rio.open(tif_file) as src:
    print(src.profile)

Output:

{'driver': 'GTiff', 'dtype': 'int16', 'nodata': -9999.0, 'width': 3660, 'height': 3660, 'count': 1, 'crs': CRS.from_epsg(32655), 'transform': Affine(30.0, 0.0, 300000.0,
       0.0, -30.0, -2700000.0), 'blockxsize': 256, 'blockysize': 256, 'tiled': True, 'compress': 'deflate', 'interleave': 'band'}

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.