[Update 7.13.2012]

My coworker noticed the the ESRI Line WKB is malformed contains unspecified trailing data, and since he's not active on GIS.SE, I posted his remarks as an answer. I'll wait a few days for critiques before accepting it in case someone has a more definitive reaction.

In the meantime, I can recreate the exception between ESRI Line Geometry WKB and SqlServer.Types by making a brand new feature class, giving it a single line geometry, and bingo---it's repeatable. Every time. But I Googled this all day yesterday and didn't find any confirmation of this. I must be a bad Googler!

[Original Post]

When I try converting an esriPolylineGeometry to IWkb, then go through the motions of exporting to wkbBytes and consuming it in SqlServer.Types.SqlGeometry.STGeomFromWKB() the conversion always fails with the error "well-known binary (WKB) input is not valid". However, if I use the same approach to get wkbBytes, it works fine when I send it through SharpMap converters. I've done everything I can imagine to rule-out poor geometries, and I'm thinking surely someone has run into this before.

On the ESRI-side, I've tried using ITopologyOperator to perform .Simplify() and .Buffer(0), and neither makes a difference. I've also tried adding .MakeValid() behind STGeomFromWKB() to no use. And there is also no difference from either IWkb.ExportToWkb() or the alternative IGeometryFactory.CreateWkbVariantFromGeometry().

So my question is: Is the WKB really invalid? If so, then why is it converting properly in SharpMap?

The WKB coming out of ESRI looks like this:


Below, the first code block shows the failing approach when I attempt to consume using SqlGeometry's static method STGeomFromWKB(). The second block shows the SharpMap approach, which works when consuming the same wkbBytes data.

This approach using SqlServer.Types fails..

// At this point, feature.Shape is esriPolylineGeometry
IWkb wkb = feature.Shape as IWkb;
int byteCount = wkb.WkbSize;
byte[] wkbBytes = new byte[byteCount];
wkb.ExportToWkb(ref byteCount, out wkbBytes[0]);

// FYI: This alternate approach to get the byte[] is no different..
//IGeometry geom = feature.Shape;
//IGeometryFactory3 geomFactory = new GeometryEnvironment() as IGeometryFactory3;
//byte[] wkbBytes = geomFactory3.CreateWkbVariantFromGeometry(wkb) as byte[];

int srid = feature.Shape.SpatialReference.FactoryCode;

// It fails here noting...
// "24115: The well-known binary (WKB) input is not valid."
SqlGeometry sqlGeometry = SqlGeometry.STGeomFromWKB(new SqlBytes(wkbBytes), srid);

string wkt = sqlGeometry.ToString();

But this approach using SharpMap converters works.. ?!?!

IWkb geomWKB = feature.Shape as IWkb;
int wkbSize = wkb.WkbSize;
byte[] geomWkbBytes = new byte[wkbSize];
wkb.ExportToWkb(ref wkbSize, out geomWkbBytes[0]);

SharpMap.Geometries.IGeometry sharpGeom = SharpMap.Converters.WellKnownBinary.GeometryFromWKB.Parse(geomWkbBytes);
string sharpWKT = SharpMap.Converters.WellKnownText.GeometryToWKT.Write(sharpGeom);

Both approaches accept Polygon geometries, but I'm hoping someone knows how to get the SqlServer.Types approach to work on Line geometries, as we have other projects already using that library. I won't hesitate to use SharpMap converters if that is the answer, but it bothers me that an unknown condition is causing SqlServer.Types to fail in the context of Line geometries. :/

  • Is the spatial reference a projected coordinate system? I would think it would fail for cases where it is a geographic coordinate system - in which case you should use SqlGeography instead of SqlGeometry. Jul 12 '12 at 22:18
  • Yes it's projected to NAD_1983_StatePlane_Missouri_Central_FIPS_2402_Feet, which returns srid=102697 from sr.FactoryCode. But I've also tried srid=0 and had the same results. :/
    – elrobis
    Jul 12 '12 at 22:21
  • You can try to decompile SQLServer.Types assembly via Reflector or other decompiler to see what causing this error.
    – megadrofan
    Jul 13 '12 at 7:21

No offense, but, actually, the conclusions of your co-worker about how something he thinks should work doesn't make it the "official way" it should work. For that, we refer to the OGC specifications.

You can download the OpenGIS Simple Features Specification for SQL 1.1 from the OGC website (a bit older) or even better, the newer OpenGIS Implementation Specification for Geographic information - Simple feature access - Common architecture 1.2.1. Either one would work.

If you refer to pages 66-70 you will find the specification for the older document, or pages 65- 72 for the newer specification.

To sum it off, there is no part that prohibits the wkb from having trailing data. In fact, the specification includes counters for every single element so you know exactly how much of the binary blob you should be parsing. It goes down to even including a byte for byte ordering so you can switch between endianness of the binary representation of every inner geometry element as you parse it!

My point is you always know exactly how much of it you should be reading and how you should be interpreting it.

Microsoft has chosen, in their parser, to continue consuming the binary content even though nothing in the OGC specification says it should. Then they chose to consider that condition "invalid". Are they wrong? Well, that is how they interpreted it even though the binary blob itself does satisfy the spec.

Saying the WKB provided by ESRI is "invalid", for this particular case, is incorrect.

By the way, every single other WKB parser out there (GEOS', PostGIS', GDAL's, ESRI's, SharpMap's, Autodesk's, etc) accepts this condition as OK.

Here is your example using PostGIS:

mydb=# select st_astext('010200000009000000BA854FAC7A273541B2325A112E643341B6FB21797E273541F080DDD356643341C87DB90187273541DA6E0FE4776433410692AC108B27354140AF88AF81643341CD1A8D1FA127354163B43190B9643341486A0F7CDF2735415B63352B7065334146AFE4D2E12735411BC18BCB776533413DA83565EC273541C03198AA9E653341861DA529F02735414B65EE25C46533410000000000');
 LINESTRING(1386362.67308842 1270830.06778256,1386366.47317479 1270870.82759863,1386375.00673662 1270903.89086049,1386379.06513322 1270913.68567939,1386401.12324684 1270969.56325843,1386463.48461022 1271152.16878339,1386465.82380195 1271159.79510123,1386476.39534999 1271198.6663848,1386480.16267571 1271236.14816888)
(1 row)


Now I am going to add any random binary data at the end of your geometry:

mydb=# select st_astext('010200000009000000BA854FAC7A273541B2325A112E643341B6FB21797E273541F080DDD356643341C87DB90187273541DA6E0FE4776433410692AC108B27354140AF88AF81643341CD1A8D1FA127354163B43190B9643341486A0F7CDF2735415B63352B7065334146AFE4D2E12735411BC18BCB776533413DA83565EC273541C03198AA9E653341861DA529F02735414B65EE25C46533411010101010101010101010101010101010101010101fff01010101fff');
 LINESTRING(1386362.67308842 1270830.06778256,1386366.47317479 1270870.82759863,1386375.00673662 1270903.89086049,1386379.06513322 1270913.68567939,1386401.12324684 1270969.56325843,1386463.48461022 1271152.16878339,1386465.82380195 1271159.79510123,1386476.39534999 1271198.6663848,1386480.16267571 1271236.14816888)
(1 row)


Still works.

The bug is actually in the Microsoft parser.

  • +1 FTW! I was hoping you would chime in on this, @Ragi. Thanks for the insightful response.
    – elrobis
    Jul 14 '12 at 22:05
  • In the spirit of good debate, though, you gotta admit those are pretty useless 0's at the end of that ESRI WKB stream. If I use the same approach to convert ESRI polygon geometry, it returns a clean stream (at least, in the eyes of the MS parser). But I fully agree it is valid where it counts. In an attempt to reach the best peace, I submit that ESRI has a dirty stream, but MS has a tyrannical parser. It's always interesting when two power forwards come together at center ice. heh
    – elrobis
    Jul 14 '12 at 22:39
  • 1
    You are right about those bytes being pretty useless. Not sure where they are coming from (some internal implementation detail probably). If you want to nitpick, I would actually say that every single byte used to describe endianness in every single geometry element is pretty dumb, too. I can see how a binary blob can come from different systems and thus serialize differently... but I think that adding it to every element is waste. I can see how you would have one at the beginning though. I do understand why it is there, but that would be a topic for a different discussion altogether. Jul 14 '12 at 23:59
  • Still a good answer after all these years. Ran into this issue with the DotSpatial ToBinary() method. github.com/DotSpatial/DotSpatial/issues/499 Jul 19 '18 at 12:41
  • Maybe technically a valid WKB, but I still call returning garbage data that is not explicitly specified as part of the WKB standard a bug. Does any other WKB serializer do this? I doubt it.
    – jpmc26
    Oct 18 '18 at 17:24

Basically my co-worker answered my main questions, so I'm punching them in here for the sake of answering the thread. Hopefully a few people will up-vote it, giving it some added credibility.

Q1: Is the ESRI Line Geometry as WKB really invalid?

[Yes.] The wkb says there is 9 points in the line, and then the parser should read 9 points. MS (rightfully) detects more data (being the 10 0's) and tells its messed up.

The 10 0's are 5 bytes in the original array. Remove the last 10 0's, and its good, like..


Q2: If the WKB is really invalid, why is the SharpMap library able to convert it?

I am 100% sure why sharplib does not fail: the format dictates that there is 9 points... if you are not picky, you just stop reading and everything SEEMS dandy. And... why would you actually validate that there are no more bytes. So im pretty sure that SharpLib does not handle it specifically.. they just stop reading after the 9 points, and everything seems to work... Thanks to MS for not just leaving Bits & Bytes hanging around.

As for WHY [the ESRI WKB is malformed] i got a few guesses: They allocate x amount of bytes at a time, and are not [precise about closing the byte array].... OR!! they , when counting the number of bytes, they take the body length and adds the header length too (1 byte encodinginfo, 1 byte geom type, 8 bytes of number of points in the line), and somehow when counting the body, they include the length of the header.. causing double header size... Anyway, thats only speculations and im not sure what the info is good for as we cant do anything about it anyway :-)

  • Ha! I love it. Thanks,@KirkKuykendall. You were able to re-create the trailing 0's, I guess?
    – elrobis
    Jul 13 '12 at 19:14
  • "Why would you actually validate that there are no more bytes?" To reduce the risk of erroneous results. You're asking the function to process some input, and the input is unexpected and seems to be self inconsistent (since its actual length doesn't match its internal counters). The authors don't know where you got this WKB; you might have hand crafted it somehow or generated it with custom code. Having self inconsistent data is likely to be indicative of a bug/mistake when creating it; validating it helps you detect that problem early rather than after your database is full of corrupt data.
    – jpmc26
    Oct 18 '18 at 17:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.