1

I would like to index Water Observations Data which stored in AWS into Open Data Cube. I tried to use

datacube product add ./wo/ga_ls_wo_3.odc-product.yaml
s3-to-dc --stac --no-sign-request "s3://dea-public-data/derivative/ga_ls_wo_3/1-6-0/088/**/*.odc-metadata.yaml" ga_ls_wo_3

The error is as follows: enter image description here

I think the error is in the odc-product.yaml file, but I don't know how to modify it. The link to the yaml file is https://explorer.sandbox.dea.ga.gov.au/products/ga_ls_wo_3.odc-product.yaml

---
# Product
# url: https://explorer.sandbox.dea.ga.gov.au/products/ga_ls_wo_3.odc-product.yaml
name: ga_ls_wo_3
license: CC-BY-4.0
metadata_type: eo3
description: Geoscience Australia Landsat Water Observations Collection 3
metadata:
  product:
    name: ga_ls_wo_3
  properties:
    odc:file_format: GeoTIFF
    odc:product_family: wo
measurements:
- name: water
  dtype: uint8
  units: '1'
  nodata: 1
  flags_definition:
    dry:
      bits:
      - 7
      - 6
      - 5
      - 4
      - 3
      - 2
      - 1
      - 0
      values:
        '0': true
      description: No water detected
    wet:
      bits:
      - 7
      - 6
      - 5
      - 4
      - 3
      - 2
      - 1
      - 0
      values:
        '128': true
      description: Clear and Wet
    cloud:
      bits: 6
      values:
        '0': false
        '1': true
      description: Cloudy
    nodata:
      bits: 0
      values:
        '0': false
        '1': true
      description: No data
    high_slope:
      bits: 4
      values:
        '0': false
        '1': true
      description: High slope
    cloud_shadow:
      bits: 5
      values:
        '0': false
        '1': true
      description: Cloud shadow
    noncontiguous:
      bits: 1
      values:
        '0': false
        '1': true
      description: At least one EO band is missing or saturated
    terrain_shadow:
      bits: 3
      values:
        '0': false
        '1': true
      description: Terrain shadow
    water_observed:
      bits: 7
      values:
        '0': false
        '1': true
      description: Classified as water by the decision tree
    low_solar_angle:
      bits: 2
      values:
        '0': false
        '1': true
      description: Low solar incidence angle
...

And the odc-metadata.yaml:

---
# Dataset
$schema: https://schemas.opendatacube.org/dataset
id: 12ed9c59-eb25-5a5b-8f78-1883ae9965e6

label: ga_ls_wo_3_088079_2015-02-21_final
product:
  name: ga_ls_wo_3
  href: https://collections.dea.ga.gov.au/product/ga_ls_wo_3

crs: epsg:32656
geometry:
  type: Polygon
  coordinates: [[[750465.9170800978, -3152883.1917287502], [569857.9783848426, -3109454.1667091143],
      [567243.3098320835, -3108799.850221707], [566133.8619656236, -3108532.052137502],
      [566092.5000000012, -3108502.5], [587032.9157076834, -3021258.993090016], [
        610792.9173861316, -2922528.9861171395], [611797.9187668706, -2918358.9803920034],
      [611827.4999999991, -2918302.4999999995], [611959.6842386074, -2918334.2145820954],
      [612322.0011885739, -2918385.8283809405], [774322.0025608335, -2957265.828710316],
      [798442.0025608329, -2963055.8287103167], [798601.2651658545, -2963114.5893311063],
      [798761.0009329331, -2963152.914271749], [798862.4999999993, -2963197.5], [
        753243.1066017176, -3153528.106601717], [753169.064409995, -3153532.101257525],
      [753135.4931736342, -3153524.058151614], [753135.0000000006, -3153525.0000000005],
      [751926.2903392814, -3153234.3534067445], [751728.9912300096, -3153187.0838449374],
      [750465.9170800978, -3152883.1917287502]]]
grids:
  default:
    shape: [7851, 7781]
    transform: [30.0, 0.0, 565785.0, 0.0, -30.0, -2918085.0, 0.0, 0.0, 1.0]

properties:
  datetime: 2015-02-21 23:35:45.162151Z
  dea:dataset_maturity: final
  dtr:end_datetime: 2015-02-21 23:36:00.043198Z
  dtr:start_datetime: 2015-02-21 23:35:30.186680Z
  eo:cloud_cover: 67.94717537706714
  eo:gsd: 30.0  # Ground sample distance (m)
  eo:instrument: OLI_TIRS
  eo:platform: landsat-8
  eo:sun_azimuth: 69.14886059
  eo:sun_elevation: 53.28764663
  fmask:clear: 1.0766182957743669
  fmask:cloud: 67.94717537706714
  fmask:cloud_shadow: 6.3372292468755
  fmask:snow: 0.0
  fmask:water: 24.638977080283002
  gqa:abs_iterative_mean_x: .nan
  gqa:abs_iterative_mean_xy: .nan
  gqa:abs_iterative_mean_y: .nan
  gqa:abs_x: .nan
  gqa:abs_xy: .nan
  gqa:abs_y: .nan
  gqa:cep90: .nan
  gqa:iterative_mean_x: .nan
  gqa:iterative_mean_xy: .nan
  gqa:iterative_mean_y: .nan
  gqa:iterative_stddev_x: .nan
  gqa:iterative_stddev_xy: .nan
  gqa:iterative_stddev_y: .nan
  gqa:mean_x: .nan
  gqa:mean_xy: .nan
  gqa:mean_y: .nan
  gqa:stddev_x: .nan
  gqa:stddev_xy: .nan
  gqa:stddev_y: .nan
  landsat:collection_category: T2
  landsat:collection_number: 1
  landsat:landsat_product_id: LC08_L1GT_088079_20150221_20170412_01_T2
  landsat:landsat_scene_id: LC80880792015052LGN01
  landsat:wrs_path: 88
  landsat:wrs_row: 79
  odc:collection_number: 3
  odc:dataset_version: 1.6.0
  odc:file_format: GeoTIFF
  odc:naming_conventions: dea_c3
  odc:processing_datetime: 2021-05-19 15:15:35.928821Z
  odc:producer: ga.gov.au
  odc:product_family: wo
  odc:region_code: '088079'

measurements:
  water:
    path: ga_ls_wo_3_088079_2015-02-21_final_water.tif

accessories:
  thumbnail:
    path: ga_ls_wo_3_088079_2015-02-21_final_thumbnail.jpg
  checksum:sha1:
    path: ga_ls_wo_3_088079_2015-02-21_final.sha1
  metadata:processor:
    path: ga_ls_wo_3_088079_2015-02-21_final.proc-info.yaml

lineage:
  ard:
  - 6dd6076f-faa0-45ff-a941-2958e8adb49b
...

1 Answer 1

0

If you want to index STAC documents, you need to change your command to the following (which selects all the .json metadata docs):

s3-to-dc \
    --stac \
    --no-sign-request \
    --skip-lineage \
    "s3://dea-public-data/derivative/ga_ls_wo_3/1-6-0/088/**/*.stac-item.json" \
    ga_ls_wo_3

You can use the command you have, just remove the --stac flag. I recommend using the STAC command above, though.

4
  • A new error occurs, it says odc.apps.dc_tools.utils.IndexingException: Failed to create dataset with error Following lineage datasets are missing from DB: 77d7551e-10a9-4598-9102-34241bc63c0b,how to fix this error?
    – Chenlu Hu
    Mar 8, 2022 at 13:16
  • Ah sorry, you need to set the flag --skip-lineage to not try to add the relationship to the parent scene.
    – Alex Leith
    Mar 9, 2022 at 21:13
  • Alex, thank you very much !!! It worked !!!
    – Chenlu Hu
    Mar 10, 2022 at 9:12
  • That's great! Please mark mine as the answer.
    – Alex Leith
    Mar 14, 2022 at 23:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.