This seems like it ought to be straightforward:
from osgeo import gdal
gpkg = gdal.OpenEx('path_to_gpk',gdal.OF_VERBOSE_ERROR,allowed_drivers=['GPKG'])
error = gpkg.DeleteLayer(0)
After running this error is always a OGRERR_FAILURE. The geopackage has been created by QGIS so I would have thought should be standard.
Can't see any way to debug further into the GDAL calls apart from a GDB session, which I might try later.