Trouble indexing data when following the cube-in-a-box installation steps

Question

I have been trying to follow the cube-in-a-box installation steps and have ran into an issue when getting to the last step where we index the data.

I have taken the following steps when installing cube-in-a-box:

1. Download cube-in-a-box from:

   https://github.com/opendatacube/cube-in-a-box.git

   And the landsat 8 data from: https://landsat.usgs.gov/sites/default/files/documents/WRS2_descending.zip

2. Unpack opendatacube-cloudformation-testing-master.zip to the desired directory (D:\Users\ehammarl\OneDrive - Capgemini\Documents\Master Thesis\ODC)

3. Navigate to the directory (D:\Users\ehammarl\OneDrive - Capgemini\Documents\Master Thesis\ODC\opendatacube-cloudformation-testing-master)

4. In 'docker-compose.yml'

   Change ODC_ACCESS_KEY and ODC_SECRET_KEY to AWS login credentials

5. Open powershell in the current directory and run 'docker-compose up'

6. Now that the current powershell window is busy, open a new powershell window in the 'opendatacube-cloudformation-testing-master'

   Now run 'docker-compose exec jupyter datacube -v system init'

   Then move the 'WRS2_descending.zip' to the directory

   D:\Users\ehammarl\OneDrive - Capgemini\Documents\Master Thesis\ODC\opendatacube-cloudformation-testing-master\data

   Next, it is time to index the data. Navigate to the /data/ folder and enter bash with 'docker-compose exec jupyter bash'

   Then cd to / with 'cd ..'

   Run 'datacube product add 'opt/odc/docs/config_samples/dataset_types/ls_usgs.yaml''

   And then 'cd /opt/odc/scripts && python3 ./autoIndex.py -p '/opt/odc/data/WRS2_descending.zip' -e '146.30,146.83,-43.54,-43.20''

After following these steps, the kernel still fails to start with the error message:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/tornado/web.py", line 1699, in _execute
    result = await result
  File "/usr/local/lib/python3.6/dist-packages/tornado/gen.py", line 742, in run
    yielded = self.gen.throw(*exc_info)  # type: ignore
  File "/usr/local/lib/python3.6/dist-packages/notebook/services/sessions/handlers.py", line 72, in post
    type=mtype))
  File "/usr/local/lib/python3.6/dist-packages/tornado/gen.py", line 735, in run
    value = future.result()
  File "/usr/local/lib/python3.6/dist-packages/tornado/gen.py", line 742, in run
    yielded = self.gen.throw(*exc_info)  # type: ignore
  File "/usr/local/lib/python3.6/dist-packages/notebook/services/sessions/sessionmanager.py", line 88, in create_session
    kernel_id = yield self.start_kernel_for_session(session_id, path, name, type, kernel_name)
  File "/usr/local/lib/python3.6/dist-packages/tornado/gen.py", line 735, in run
    value = future.result()
  File "/usr/local/lib/python3.6/dist-packages/tornado/gen.py", line 742, in run
    yielded = self.gen.throw(*exc_info)  # type: ignore
  File "/usr/local/lib/python3.6/dist-packages/notebook/services/sessions/sessionmanager.py", line 101, in start_kernel_for_session
    self.kernel_manager.start_kernel(path=kernel_path, kernel_name=kernel_name)
  File "/usr/local/lib/python3.6/dist-packages/tornado/gen.py", line 735, in run
    value = future.result()
  File "/usr/local/lib/python3.6/dist-packages/tornado/gen.py", line 209, in wrapper
    yielded = next(result)
  File "/usr/local/lib/python3.6/dist-packages/notebook/services/kernels/kernelmanager.py", line 168, in start_kernel
    super(MappingKernelManager, self).start_kernel(**kwargs)
  File "/usr/local/lib/python3.6/dist-packages/jupyter_client/multikernelmanager.py", line 110, in start_kernel
    km.start_kernel(**kwargs)
  File "/usr/local/lib/python3.6/dist-packages/jupyter_client/manager.py", line 240, in start_kernel
    self.write_connection_file()
  File "/usr/local/lib/python3.6/dist-packages/jupyter_client/connect.py", line 547, in write_connection_file
    kernel_name=self.kernel_name
  File "/usr/local/lib/python3.6/dist-packages/jupyter_client/connect.py", line 212, in write_connection_file
    with secure_write(fname) as f:
  File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/usr/local/lib/python3.6/dist-packages/jupyter_client/connect.py", line 105, in secure_write
    assert '0600' == oct(stat.S_IMODE(os.stat(fname).st_mode)).replace('0o', '0')
AssertionError

When reading through the logs of each step, there is only one error:

2020-01-27 14:10:37,715 INFO Indexing 5c7e2961-3c54-5738-bbea-b0ed1e56b708
Process Process-2:
Traceback (most recent call last):
  File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "./ls_public_bucket.py", line 227, in worker
    data = make_metadata_doc(txt_doc, bucket_name, key)
  File "./ls_public_bucket.py", line 160, in make_metadata_doc
    } for band in bands
  File "./ls_public_bucket.py", line 160, in <dictcomp>
    } for band in bands
KeyError: 'FILE_NAME_BAND_10'

Is there something that I am doing wrong in the steps that I took to install cube-in-a-box or is there something else that needs to be configured to get up and running?

Answer 1

I found a solution to the problem, it turns out that the latest version of jupyter_client breaks the environment on a server.

So after rolling back the jupyter_client to version 5.3.1 with:

docker-compose exec jupyter pip install --user jupyter_client==5.3.1

Everything works as it should.