How can a single product be dumped from the index database to be included in another index database? I know a few Postgres commands to dump the full agdc database within the Postgres database, but the indices we are working with are becoming quite large, so being able to restore individual products without losing others (not a database overwrite - additive) would be quite useful.
2 Answers
You are looking to transfer products across datacubes ? Unfortunately due to how databases function and indexes being an instance specific optimisation the db server makes there is no way to transport indexes across.
If indexes are becoming an issue in a large DB it makes sense to move them to their dedicated storage media and optimise IOPs for that. See details here : https://www.cybertec-postgresql.com/en/improving-transaction-latency-by-moving-indexes-to-faster-media/
I have learnt this from bitter personal experience transporting datacubes via db dump from NCI to AWS (20+ million rows of datasets).
That said we do have some tools for persisting a collection of datasets to a file based DB. Please see my DB backend enhancement proposal here : https://github.com/opendatacube/datacube-core/wiki/ODC-EP-004---Use-alternative-index-backends#lmdb
As @whatnick says, you can do a kind of ETL between databases, taking care to retain product and dataset relations.
Although I haven't used it, this will export metadata from an existing database.
And if you can make indexing fast enough, just doing it again is a good bet!
