0

Indexing USGS Landsat Collection 2 Level 2 datasets into Open Data Cube

Following the suggestion in the above URL, I tried to index USGS data into the Open Data Cube. Product definition was successfully added, however when I tried to index using stac-to-dc, I received the following error:

command used:

docker-compose exec jupyter stac-to-dc --catalog-href=https://landsatlook.usgs.gov/stac-server/ --rewrite-assets=https://landsatlook.usgs.gov/data/,s3://usgs-landsat/ --bbox=91.81029362088168,21.371284249219112,92.09814642753065,21.537368083932826 --collections=landsat-c2l2-sr --datetime=2021-01-01/2021-01-31

exception:

Indexing from STAC API...
02/21/2022 04:03:36: ERROR: Failed to handle item LE07_L2SP_136045_20210113_20210208_02_T1_SR with exception Failed to create dataset with error No matching Product found for dataset 0e138867-8b65-5712-9216-395e7af280ad
 The URI was https://landsatlook.usgs.gov/stac-server/collections/landsat-c2l2-sr/items/LE07_L2SP_136045_20210113_20210208_02_T1_SR
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/odc/apps/dc_tools/stac_api_to_dc.py", line 158, in stac_api_to_odc
    _ = future.result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 437, in result
    return self.__get_result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/dist-packages/odc/apps/dc_tools/stac_api_to_dc.py", line 106, in process_item
    index_update_dataset(
  File "/usr/local/lib/python3.8/dist-packages/odc/apps/dc_tools/utils.py", line 172, in index_update_dataset
    raise IndexingException(
odc.apps.dc_tools.utils.IndexingException: Failed to create dataset with error No matching Product found for dataset 0e138867-8b65-5712-9216-395e7af280ad
 The URI was https://landsatlook.usgs.gov/stac-server/collections/landsat-c2l2-sr/items/LE07_L2SP_136045_20210113_20210208_02_T1_SR
02/21/2022 04:03:36: ERROR: Failed to handle item LC08_L2SP_136045_20210121_20210307_02_T1_SR with exception Failed to create dataset with error No matching Product found for dataset 7c767047-c6fe-5ca1-84b2-82a671331700
 The URI was https://landsatlook.usgs.gov/stac-server/collections/landsat-c2l2-sr/items/LC08_L2SP_136045_20210121_20210307_02_T1_SR
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/odc/apps/dc_tools/stac_api_to_dc.py", line 158, in stac_api_to_odc
    _ = future.result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 437, in result
    return self.__get_result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/dist-packages/odc/apps/dc_tools/stac_api_to_dc.py", line 106, in process_item
    index_update_dataset(
  File "/usr/local/lib/python3.8/dist-packages/odc/apps/dc_tools/utils.py", line 172, in index_update_dataset
    raise IndexingException(
odc.apps.dc_tools.utils.IndexingException: Failed to create dataset with error No matching Product found for dataset 7c767047-c6fe-5ca1-84b2-82a671331700
 The URI was https://landsatlook.usgs.gov/stac-server/collections/landsat-c2l2-sr/items/LC08_L2SP_136045_20210121_20210307_02_T1_SR
02/21/2022 04:03:36: ERROR: Failed to handle item LE07_L2SP_135045_20210122_20210217_02_T1_SR with exception Failed to create dataset with error No matching Product found for dataset 80c2ed31-24e1-5e39-a075-290db3d231b4
 The URI was https://landsatlook.usgs.gov/stac-server/collections/landsat-c2l2-sr/items/LE07_L2SP_135045_20210122_20210217_02_T1_SR
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/odc/apps/dc_tools/stac_api_to_dc.py", line 158, in stac_api_to_odc
    _ = future.result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 437, in result
    return self.__get_result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/dist-packages/odc/apps/dc_tools/stac_api_to_dc.py", line 106, in process_item
    index_update_dataset(
  File "/usr/local/lib/python3.8/dist-packages/odc/apps/dc_tools/utils.py", line 172, in index_update_dataset
    raise IndexingException(
odc.apps.dc_tools.utils.IndexingException: Failed to create dataset with error No matching Product found for dataset 80c2ed31-24e1-5e39-a075-290db3d231b4
 The URI was https://landsatlook.usgs.gov/stac-server/collections/landsat-c2l2-sr/items/LE07_L2SP_135045_20210122_20210217_02_T1_SR
02/21/2022 04:03:36: ERROR: Failed to handle item LE07_L2SP_135045_20210106_20210201_02_T1_SR with exception Failed to create dataset with error No matching Product found for dataset a55a7a56-2880-52d9-ab3b-24a62aeeca30
 The URI was https://landsatlook.usgs.gov/stac-server/collections/landsat-c2l2-sr/items/LE07_L2SP_135045_20210106_20210201_02_T1_SR
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/odc/apps/dc_tools/stac_api_to_dc.py", line 158, in stac_api_to_odc
    _ = future.result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 437, in result
    return self.__get_result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/dist-packages/odc/apps/dc_tools/stac_api_to_dc.py", line 106, in process_item
    index_update_dataset(
  File "/usr/local/lib/python3.8/dist-packages/odc/apps/dc_tools/utils.py", line 172, in index_update_dataset
    raise IndexingException(
odc.apps.dc_tools.utils.IndexingException: Failed to create dataset with error No matching Product found for dataset a55a7a56-2880-52d9-ab3b-24a62aeeca30
 The URI was https://landsatlook.usgs.gov/stac-server/collections/landsat-c2l2-sr/items/LE07_L2SP_135045_20210106_20210201_02_T1_SR
02/21/2022 04:03:36: ERROR: Failed to handle item LC08_L2SP_136045_20210105_20210309_02_T1_SR with exception Failed to create dataset with error No matching Product found for dataset 07e8baa2-50ea-5a22-bcb1-bf3a74524a87
 The URI was https://landsatlook.usgs.gov/stac-server/collections/landsat-c2l2-sr/items/LC08_L2SP_136045_20210105_20210309_02_T1_SR
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/odc/apps/dc_tools/stac_api_to_dc.py", line 158, in stac_api_to_odc
    _ = future.result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 437, in result
    return self.__get_result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/dist-packages/odc/apps/dc_tools/stac_api_to_dc.py", line 106, in process_item
    index_update_dataset(
  File "/usr/local/lib/python3.8/dist-packages/odc/apps/dc_tools/utils.py", line 172, in index_update_dataset
    raise IndexingException(
odc.apps.dc_tools.utils.IndexingException: Failed to create dataset with error No matching Product found for dataset 07e8baa2-50ea-5a22-bcb1-bf3a74524a87
 The URI was https://landsatlook.usgs.gov/stac-server/collections/landsat-c2l2-sr/items/LC08_L2SP_136045_20210105_20210309_02_T1_SR
02/21/2022 04:03:36: ERROR: Failed to handle item LE07_L2SP_136045_20210129_20210224_02_T1_SR with exception Failed to create dataset with error No matching Product found for dataset 0eb2aca9-0445-5675-b6c6-2853fed3358c
 The URI was https://landsatlook.usgs.gov/stac-server/collections/landsat-c2l2-sr/items/LE07_L2SP_136045_20210129_20210224_02_T1_SR
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/odc/apps/dc_tools/stac_api_to_dc.py", line 158, in stac_api_to_odc
    _ = future.result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 437, in result
    return self.__get_result()
  File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/dist-packages/odc/apps/dc_tools/stac_api_to_dc.py", line 106, in process_item
    index_update_dataset(
  File "/usr/local/lib/python3.8/dist-packages/odc/apps/dc_tools/utils.py", line 172, in index_update_dataset
    raise IndexingException(
odc.apps.dc_tools.utils.IndexingException: Failed to create dataset with error No matching Product found for dataset 0eb2aca9-0445-5675-b6c6-2853fed3358c
 The URI was https://landsatlook.usgs.gov/stac-server/collections/landsat-c2l2-sr/items/LE07_L2SP_136045_20210129_20210224_02_T1_SR
Added 0 Datasets, failed 6 Datasets

Any thoughts?

Improve this question
2

1 Answer 1

Reset to default
0

I was able to do this just now.

I ran this to add the products:

datacube product add https://raw.githubusercontent.com/opendatacube/datacube-dataset-config/main/products/lsX_c2l2_sr.odc-product.yaml

And this to do the indexing:

stac-to-dc --catalog-href=https://landsatlook.usgs.gov/stac-server/ --rewrite-assets=https://landsatlook.usgs.gov/data/,s3://usgs-landsat/ --bbox=91.81029362088168,21.371284249219112,92.09814642753065,21.537368083932826 --collections=landsat-c2l2-sr --datetime=2021-01-01/2021-01-31

With the output:

Indexing from STAC API... Added 6 Datasets, failed 0 Datasets

Improve this answer
1
  • I had the issue only from the cube-in-a-box which I have deployed middle of last year. However, with a clean deployment, it works as you have suggested. Many thanks for your kind help.
    – prpp2021
    Feb 22, 2022 at 10:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.