The GeoPackage standard does not have anything defined about how to save the styles yet. Styles are coming to the standard but it is hard to say when. Here is a report about one experiment that has been done for OGC http://docs.opengeospatial.org/per/16-037.pdf.
What you can do with GDAL is to add a new field "OGR_STYLE" into the feature table and write ...
GDAL itself will be installed on Windows under C:\Program Files\GDAL. However, the python bindings will be installed under the relevant python directory that is recorded in your registry. If you have separately installed Python before or after installing ArcGIS then you will have at least two python versions. This isn't a problem necessarily except, when ...
You need to create a shapefile with precision. The DBF format (which is what the shapefile uses to store attribute information) can store decimal places as part of the numeric data type (see here) so your problem is probably that you've not told pygdal what precision you want on your numeric.
Proper code below:
from osgeo import gdal
gpkg = gdal.OpenEx('path_to_gpk',gdal.OF_OF_UPDATE,allowed_drivers=['GPKG'])
error = gpkg.DeleteLayer(0)
# this code is also possible as an alternative
# doesn't seem to delete everything though, so possibly DeleteLayer() is better
# gpkg.ExecuteSQL('DROP TABLE [layer name]')
The solution turned out to be quite simple. I had deeper look into the options of ogr2ogr and discovered the
Layer creation option (format specific)
Since I'm creating av shapefile, I had a look at GDAL's ESRI Shapefile format page. In the section "Layer Creation Options" I found
SHPT=type: Override the type of shapefile created....