pdal | issues when using tiledb reader in python

Question

I have created an fresh conda environment and installed pdal per the pdal Quickstart instructions:

conda create --yes --name pdalplayground --channel conda-forge pdal

All applications (tindex, tile, info, pipeline...etc) when run from the environment's command line work as expected.

I am working on a simple pipeline, where the first stage is reading from tiledb. When I "hard-code" in various parameters in this pipeline (pipeline.json), and run pipeline application from command line - everything works.

So then I tried reproducing pipeline in a python script (below). At first, I got this error. I had installed python-pdal from conda-forge as well.

RuntimeError: Couldn't create reader stage of type 'readers.tiledb'.

Poking through the pdal FAQ - it looks like I need to update the search path for plugins. I have done this at top of script - just pointing to the bin directory created from conda pdal package.

os.environ["PDAL_DRIVER_PATH"] = r'C:\Users\hodgest\.conda\envs\pdalplayground\Library\bin'

After setting PDAL_DRIVER_PATH - the tiledb reader works in the python script. So then I added a reprojection stage. Sometimes the script below totally crashes the python process, other times it completes just fine. I am not modifying the underlying tiledb array at all.

Am I doing something obviously wrong here?

import json
import pdal
import os

os.environ["PDAL_DRIVER_PATH"] = r'C:\Users\hodgest\.conda\envs\pdalplayground\Library\bin'

PIPELINE = [
    {
        "type": "readers.tiledb",
        "array_name": "barrylidar"
    },
    {
        "type": "filters.reprojection",
        "in_srs": "EPSG:32615",
        "out_srs": "EPSG:4326"
    }
]

pipeline = pdal.Pipeline(json.dumps(PIPELINE))
pipeline.validate()

# set pdal log level
pipeline.loglevel = 8

# execute the pipeline
count = pipeline.execute()
arrays = pipeline.arrays
metadata = pipeline.metadata
log = pipeline.log
print("done")

Answer 1

validate does not need to be called on the pipeline to run it and if you comment out pipeline.validate() then the pipeline executes within Python.

This is a bug https://github.com/PDAL/PDAL/issues/2891 where calling validate as it clears the metadata for the stage. This causes an issue with the TileDB driver in that metadata is read/written and required. The segfault happens because of the missing metadata. The PR linked to in the issue fixes this in the TileDB driver when the metadata is changed by calling validate.