I am working on a piece of code that does a pretty complex computation using GPKG layer feature areas, I then calculate several new fields . Finally, when I try and write the results back to disk, I get an inconsistent (doesn't always appear) and nondescript error from what I assume is OGR: 'Incompatible geometry for operation'.
I am working with a LOT of features, so I am utilizing multi-threaded processing. All of my vector data is stored in zip files, so I use temporary directories to unzip, process and then re-zip the data. The line of code where I think I get the error looks like this:
dst_vector_disk_ds = ogr.GetDriverByName('GPKG').CopyDataSource(out_mem_ds, vector_file.replace('features', f'split_{job_index}'))
where vector_file is the original vector file name (string), out_mem_ds is the in-memory data set with all the processed features now neatly packaged away in their respective layer and job_index is the thread number referring to the multi-threaded processing.
I am using WSL (Windows 10), Anaconda3, Python 3.8.2 environment, GDAL version 3.0.2 and I have already used the flag ogr.UseExceptions().
Haven't been able to pinpoint the actual source of this error and since my Python code actually calls on a .c file module (compiled with Cython), de-bugging this has been very difficult.
The vector data is a layer of polygons. The threads write temporary outputs and then pass those back to the parent which merges all the resulting temporary output files to a permanent file.I have narrowed the source after a lot of manual de-bugging: The incompatible geometries come from the in-memory features that I am trying to write out to disk not having their original geometries associated. I have tried storing their geometries and re-associating them, but I ran into one of the classic ogr reference pitfalls/gotchas as mentioned in the GDAL Gotchas.
