After making this post: layer.materialize() function not copying all features within layer
I ran another multi-part test to determine why layer.materialize() does not copy all features within geopackage layers.
Result: I determined geopackage layers (obtained from converting s57 layers to a gpkg file) containing features that have non NULL json values will not be copied via layer.materialize(). Determining why this is the case is the reason for this post. A possible explanation is at the bottom of this post.
I am using Qgis 3.40.1 I am using Linux (Rocky 8)
Steps of the Test:
1: video for step 1: https://jmp.sh/j2XzZpT1
Load layers from an s57 file (I chose US2AK30M.000). Then convert the "LIGHTS" layer to a geopackage layer (I chose the LIGHTS layer because it contains features with non NULL values for "JSON" fields.
2: video for steps 2-8: https://jmp.sh/cnEc7KVr Remove all other layers except the geopackage LIGHTS layer and the s57 LIGHTS layer
3: Runn this python script to create memory layer copies from the above layers:
for curLayer in QgsProject.instance().mapLayers().values():
request = QgsFeatureRequest().setFilterFids(curLayer.allFeatureIds())
print(request.filterFids())
clone = curLayer.materialize(request)
print(clone)
QgsProject.instance().addMapLayer(clone)
4: Observe how the memory layer created from the s57 layer contains all the features, but the memory layer created from the gpkg layer contains none of the features.
5: Now delete the created memory layers and open the attribute table for the gpkg layer.
6: Delete all json value fields.
7: Close the attribute table (dont save your changes/cancel the editing of the gpkg layer) and run the python script again.
8: Observe that the memory layer created from the geopackage layer contains all of the features.
9: Now, cancel the editing stage of the gpkg file do not save your changes. We want to get back the json fields we deleted earlier.
10: Now, delete only the following json fields that contain features with non NULL values: CATLIT, COLOUR, LITVIS, STATUS (If you have a sql browser, note that values in these columns are json arrays, not json objects as shown in this video: https://jmp.sh/4Xm19123 ). Again do not save your edits.
11: Now run the python script again
12: Observe that the memory layer created from the geopackage layer contains all of the features, even though the LNAM, LNAM_REFS, and FFPT_RIND columns remain.
I am thinking the reason these features are unable to be copied is because, even though the problematic column's of the geopackage layer file say JSON, the values in the columns are not key value pairs of a object. In the attribute viewer, the values are arrays of strings in the s57 layer, but they become json values after being loaded into a geopackage file.
Viewing the geopackage file itself with a sql browser (instead of looking at the attribute viewer of the geopackage file in qgis), it shows that the type of the problematic columns are listed as TEXT. The values are TEXT (strings), but they are strings holding json arrays. Maybe qgis has trouble loading strings containing json arrays? I will have to do more testing.
