1

I am trying to index sentinel-l2a-cogs following the guide here. The product definition is added to the database.

(cubeenv) λ datacube product list
s2_l2a  Sentinel-2a and Sentinel-2b imagery, processed to Level 2A (Surface Reflectance) and converted to Cloud Optimized GeoTIFFs

but I get the error on running s3-to-dc --stac --no-sign-request s3://deafrica-sentinel-2/sentinel-s2-l2a-cogs/37/M/CS/2017/10/**/S2A_37MCS_20171016_0_L2A.json s2_l2a

Traceback (most recent call last):
  File "D:\anaconda\envs\cubeenv\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "D:\anaconda\envs\cubeenv\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "D:\anaconda\envs\cubeenv\Scripts\s3-to-dc.exe\__main__.py", line 7, in <module>
  File "D:\anaconda\envs\cubeenv\lib\site-packages\click\core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "D:\anaconda\envs\cubeenv\lib\site-packages\click\core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "D:\anaconda\envs\cubeenv\lib\site-packages\click\core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "D:\anaconda\envs\cubeenv\lib\site-packages\click\core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "D:\anaconda\envs\cubeenv\lib\site-packages\odc\apps\dc_tools\s3_to_dc.py", line 129, in cli
    added, failed = dump_to_odc(
  File "D:\anaconda\envs\cubeenv\lib\site-packages\odc\apps\dc_tools\s3_to_dc.py", line 56, in dump_to_odc
    for uri, metadata in uris_docs:
  File "D:\anaconda\envs\cubeenv\lib\site-packages\odc\apps\dc_tools\_docs.py", line 89, in parse_doc_stream
    for uri, doc in doc_stream:
  File "D:\anaconda\envs\cubeenv\lib\site-packages\odc\apps\dc_tools\s3_to_dc.py", line 31, in stream_docs
    for document in documents:
  File "D:\anaconda\envs\cubeenv\lib\site-packages\odc\aio\__init__.py", line 494, in __call__
    for rr, ee in future_results(
  File "D:\anaconda\envs\cubeenv\lib\site-packages\odc\ppt\__init__.py", line 85, in future_results
    have_more = fill(it, need_n, active)
  File "D:\anaconda\envs\cubeenv\lib\site-packages\odc\ppt\__init__.py", line 70, in fill
    x = next(src)
  File "D:\anaconda\envs\cubeenv\lib\site-packages\odc\aio\__init__.py", line 486, in generate_requests
    for url in urls:
  File "D:\anaconda\envs\cubeenv\lib\site-packages\odc\apps\dc_tools\s3_to_dc.py", line 25, in stream_urls
    for url in urls:
  File "D:\anaconda\envs\cubeenv\lib\site-packages\odc\aio\__init__.py", line 433, in find
    raise IOError(f"Failed to list: {url}")
OSError: Failed to list: s3://deafrica-sentinel-2/sentinel-s2-l2a-cogs/37/M/CS/2017/10/

On running datacube system check I get the following output.

Version:       1.8.6
Config files:  C:\Users\91911\.datacube_integration.conf
Host:          localhost:5432
Database:      opendatacube
User:          postgres
Environment:   None
Index Driver:  default

Valid connection:       YES

I am running in cubeenv conda environment and I am able to connect to the database as well. Additionally my DATACUBE_CONFIG_PATH is pointing to datacube_integration.conf. What is causing the problem and how do I resolve the issue? I was able to downloaded the STAC document manually using the following command

aws s3 cp --no-sign-request --region af-south-1 s3://deafrica-sentinel-2/sentinel-s2-l2a-cogs/37/M/CS/2017/10/S2A_37MCS_20171003_0_L2A/S2A_37MCS_20171003_0_L2A.json ./

1 Answer 1

3

I think there was a minor issue with the docs, which I have fixed.

Try indexing from the Element 84-managed bucket in us-west-2, like this:

s3-to-dc --stac --no-sign-request \
"s3://sentinel-cogs/sentinel-s2-l2a-cogs/37/M/CS/**/*S2A_37MCS_20171016_0_L2A*.json" s2_l2a

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.