Well I think I have a solution and the answer is that the polygon is and isn't topologically correct.
If I use the ArcObjects ITopologicalOperator4.IsSimple operator it results in the geometry being simple which corresponds to a valid polygon. The documentation says "Indicates whether this geometry is known (or assumed) to be topologically correct, after explicitly determining this if the geometry is not already known (or assumed) to be simple."
If I use the ArcMap Check Geometry function that results in no issues. This reports that the geometry is valid.
The reason I think these two functions result in a valid geometry is because they are using the ESRI validation method and not the OGC validation method. To test this I used the ArcGIS Pro Check Geometry function which allows you to select a validation method: ESRI OR OGC. The ESRI method passes and the OGC method returns Problem=non simple.
So, to get this working in ArcMap I used the DotSpatial libraries since they use NetTopologySuite which is OGC compliant and I don't think ESRI will add this to ArcObjects (only bugs are being fixed and no new functionality).
Here is my DotSpatial vs ArcObjects test
'first we take the ArcObjects polygon and create a WKB representation
Dim geomTemp As IGeometry = pPolygonFeature.ShapeCopy
Dim wkb As IWkb = CType(geomTemp, IWkb)
Dim wkbBytes() As Byte = New Byte(wkb.WkbSize - 1) {}
Dim byteCount As Integer = wkb.WkbSize
wkb.ExportToWkb(byteCount, wkbBytes(0))
'now we convert the WKB to a DotSpatial object
Dim wr As New NetTopologySuite.IO.WKBReader
Dim dsGeom As GeoAPI.Geometries.IGeometry = wr.Read(wkbBytes)
Dim IsValidDsGeom As Boolean = dsGeom.IsValid
Debug.Print("IsValidDsGeom={0}", IsValidDsGeom)
'check if the polygon is simple
Dim pTopo As ITopologicalOperator4 = CType(pPolygonGeom, ITopologicalOperator4)
Debug.Print("PolygonGeom is Simple={0}", pTopo.IsSimple)
Dim reas As esriNonSimpleReasonEnum
Dim IsSimpEx As Boolean = pTopo.IsSimpleEx(reas)
Debug.Print("PolygonGeom is SimpleEx={0} with reason={1}", IsSimpEx, reas)
Debug.Print("PolygonGeom is KnownSimple={0}", pTopo.IsKnownSimple)
The Debug output:
IsValidDsGeom=False
PolygonGeom is Simple=True
PolygonGeom is SimpleEx=True with reason=esriNonSimpleOK
PolygonGeom is KnownSimple=True