Using rasterio to read NDVI data from GIMMS's COG products

Question

I try to retrieve some NDVI data from GIMMS' cloud-optimised geotiffs using rasterio.

However, when I use the following code I always get back this 403 error: rasterio.errors.RasterioIOError: HTTP response code: 403

import rasterio

filepath = "https://gimms.gsfc.nasa.gov/MODIS/std/GMOD09Q1/cog/NDVI/2022/305/GMOD09Q1.A2022305.08d.latlon.global.061.NDVI.tif"
with rasterio.open(filepath) as src:
    print(src.profile)

Hence, my question is this: am I doing something wrong or it is not possible to retrieve data from this service like this?

Answer 1

No you're not doing anything wrong. Here's an example showing a working COG and the non-working COG:

import rasterio

filepath = "https://ss6imagery.arcgisonline.com/imagery_sample/landsat8/Bolivia_LC08_L1TP_001069_20190719_MS.tiff"
print(filepath)
with rasterio.open(filepath) as src:
    print(src.profile)

filepath = "https://gimms.gsfc.nasa.gov/MODIS/std/GMOD09Q1/cog/NDVI/2022/305/GMOD09Q1.A2022305.08d.latlon.global.061.NDVI.tif"
print(filepath)
with rasterio.open(filepath) as src:
    print(src.profile)

Output:

https://ss6imagery.arcgisonline.com/imagery_sample/landsat8/Bolivia_LC08_L1TP_001069_20190719_MS.tiff
{'driver': 'GTiff', 'dtype': 'uint16', 'nodata': 0.0, 'width': 7661, 'height': 7781, 'count': 8, 
 'crs': CRS.from_epsg(32619), 'transform': Affine(30.0, 0.0, 561885.0, 0.0, -30.0, -1322685.0), 
 'blockxsize': 512, 'blockysize': 512, 'tiled': True, 'compress': 'lzw', 'interleave': 'pixel'}

https://gimms.gsfc.nasa.gov/MODIS/std/GMOD09Q1/cog/NDVI/2022/305/GMOD09Q1.A2022305.08d.latlon.global.061.NDVI.tif
...
rasterio.errors.RasterioIOError: HTTP response code: 403

It's just that the service refuses:

• requests without a user agent.
• range requests

However, you might be able access it using the GDAL /vsicurl_streaming virtual filesystem and setting a user agent:

import rasterio

filepath = "/vsicurl_streaming/https://gimms.gsfc.nasa.gov/MODIS/std/GMOD09Q1/cog/NDVI/2022/305/GMOD09Q1.A2022305.08d.latlon.global.061.NDVI.tif"
print(filepath)
with rasterio.Env(GDAL_HTTP_USERAGENT="rasterio"):
    with rasterio.open(filepath) as src:
        print(src.profile)

Output:

/vsicurl_streaming/https://gimms.gsfc.nasa.gov/MODIS/std/GMOD09Q1/cog/NDVI/2022/305/GMOD09Q1.A2022305.08d.latlon.global.061.NDVI.tif
{'driver': 'GTiff', 'dtype': 'uint8', 'nodata': None, 'width': 160000, 'height': 64000, 'count': 1, 'crs': CRS.from_epsg(4326), 'transform': Affine(0.00225, 0.0, -180.0,
       0.0, -0.00225, 81.0), 'blockxsize': 256, 'blockysize': 256, 'tiled': True, 'compress': 'lzw', 'interleave': 'band'}

But note that if you need a window, rasterio will need to download everything up to that window as the server refuses range requests, which pretty much defeats the purpose of a "Cloud Optimized GeoTIFF".

Might be easier to download it locally, then access it.