32

I've been writing a shapefile parsing library, and have encountered a couple of design decisions in the specification that I don't immediately understand. I'm hoping there's a wizened old ESRI developer around here who can tell me why these things are the way they are.

  1. The main record file (.shp) is of mixed endianness. Specifically, parts of the header features big endian byte ordering, but the records are all little endian. I typically work at a higher level than bytes and bits, but everything I've so far read about endianness marks this as unusual. Why isn't the file specified to be of uniform endianness?

  2. The "File Length" field, as well as other length and position fields, are recorded in 16-bit words, instead of the more standard (from my limited perspective) 8 bit positioning. How did this decision get reached?

I posted a similar question on Stack Overflow, but didn't get any response. If this seems too off topic to other people, I could support closing it.

Improve this question
2
  • 4
    Joel Lawhead at GeospatialPython.com has been working on solving shapefile mysteries for a while.
    – Chad Cooper
    Commented Jan 18, 2012 at 16:01
  • Not exactly related, but neat! I hope the figure it out.
    – canisrufus
    Commented Jan 18, 2012 at 18:21

6 Answers 6

Reset to default
28

The development of shapefiles was concurrent with the development of ArcView, which was specifically designed to be platform independent. (In fact, that turned out to be its downfall: by relying on an interface developed in a platform independent GUI called "Neuron Data," it could not take advantage of many Windows capabilities. It ended up reflecting the worst of all the systems it was marketed for.) Although the shapefile specification was weird from the beginning, it made a loopy sort of sense within this design framework: because shapefiles were intended for many platforms, their specification should not favor any one of them and therefore should be equally obnoxious to programmers of all persuasions.

The second question appears to be based on an assumption that is not true. For instance, the "File Length" field appears at byte offset 24 in the main header and is a (signed) four-byte (32 bit) integer, as it must be in order to represent a length of up to 2^31-1. It is preceded by a four-byte "File Code" and five more four-byte fields reserved for future use: when you're reserving such space, of course you want to make the fields as large as reasonably possible, which at the time was 32 bits, in order to maintain the greatest possible flexibility. It helps, too, to align numeric fields in a file on word boundaries: machine-level code to parse them is a little easier to write and it can avoid potential (subtle) problems with upper-level compilers that might automatically pad their STRUCTs to align with words or doublewords.

Improve this answer
8
  • 2
    :) Exactly what I was looking for. When I say that the "File Length" field is "recorded in 16-bit words", what I'm attempting to say is that the value of the 32-bit integer records the file length in 16-bit words. (From the spec: "The value for file length is the total length of the file in 16-bit words"). It seems like it can represent a byte length of 2*2^31-1, which looks to be about 4 GB. The same is true for the values in the .shx file. It looks like it should be able to support file lengths up to 2*2^31-1 bytes. What am I missing?
    – canisrufus
    Commented Jan 18, 2012 at 17:28
  • Good point--I missed that. Actually, the design could just as easily have made file lengths and offsets (pointers in the .shx file) in terms of four-byte words, thereby increasing the possible size of the .shp file to 4*(2^31-1) (about 8 billion bytes). I have no idea why they chose two-byte words, nor even why they consistently use signed integers where unsigned integers are both more appropriate and provide for twice as much storage.
    – whuber
    Commented Jan 18, 2012 at 21:08
  • 1
    I wonder if the 16-bit oddity has to do with 16-bit computers used at the time, where a native int was 16-bits.
    – Mike T
    Commented Jan 18, 2012 at 22:54
  • It's always a possibility, @Mike. However, even the 80286 PCs (c. 1984) natively supported 32-bit ints--they used register pairs to do arithmetic with them.
    – whuber
    Commented Jan 18, 2012 at 22:57
  • 6
    An Esri colleague says that he remembers the mix of endian-ness was deliberate. Something along the lines of 'we'll make developers handle it outright because of cross-platform issues.' But, of course, this is all apocryphal.
    – mkennedy
    Commented Jan 20, 2012 at 15:33
11

Somebody out there knows these answers and more but they ain't talking.

The team I've been working with to decode the undocumented sbn and sbx files has discovered many more oddities that are both similar yet even more bizarre at the same time.

Most of the shapefile structures are logical and very efficient which suggest the ESRI developers thought things through. It's like they had a bunch of smart developers with one lunatic thrown in.

As suggested by other posts the oddities are probably the result of machine or language requirements that are foreign to us now.

I always suspected the 16-bit words were an easy way to save space. You'll find that you have to hold the 16-bit word values in memory when handling files. The strategy of calculating values to save space is common in binary formats even today. But Mike's native int suggestion is also just as likely.

The endian-flipping is just weird. Nobody has a good answer that I've seen.

The dbf format was ripped from the dbase III format originated in the 1960's. It has been widely used ever since and can be found under other names including foxpro and xbase.

Despite the shapefile format's flaws, oddities, and limitations it persists stubbornly in and around the field of GIS. Every other attempt to replace it has been too bloated for simple vector storage or too proprietary. Even ESRI thought shapefiles would be a toy that would move beginners towards ArcINFO, coverages, and geodatabases. The Internet probably had a lot to do with the format taking off.

I learned a lot writing pyshp. Writing a parser is a fantastic way to learn a format.

Improve this answer
6
  • Hmm. Good answer. I don't understand how the use of 16-bit words saves space. For my purposes (building ArrayBufferViews in javascript), all it does is force me to multiply by two to get the correct offset: I'm burning extra cycles for no benefit. Would you elaborate?
    – canisrufus
    Commented Jan 19, 2012 at 13:55
  • 1
    Yes - since they used signed ints they're upper end on those values would be 32,767 so they can store larger numbers in 2-bytes instead of 4. The values assigned to 16-bit words as I said are values you end up holding in RAM when working with shapefiles for read and write operations. Coming up with a scheme to save space on doubles (which I have seen in other binary formats) is always ugly and complicated. So they just stuck with a simple scheme for data size values.
    – GeospatialPython.com
    Commented Jan 19, 2012 at 17:28
  • Also - I discovered in the shx files which stumped me at first. SHX files have bounding boxes for features mapped to a 256x256 integer grid. This technique is common in indexing but not on a grid that small. They save the coordinates as 1-byte chars instead of ints. That's why the grid is only 256x256. Now that's being downright stingy with memory even for the 1990's! There are of course many other efficiencies like the implied grouping of parts using an index. You're right - these techniques put more burden on the programmer. So memory usage must have been a priority.
    – GeospatialPython.com
    Commented Jan 19, 2012 at 17:38
  • 1
    Yah, I read your write up. You're doing the lord's good work on that one ;) I'm eagerly awaiting your final analysis. Regarding the 16-bit issue, I'm not sure your point holds. 1. In the SHP and SHX files, there are no 16 bit fields, unless I'm sorely mistaken. 2. Representing 16-bit values instead of 8-bit values only doubles the describable length (2*2^15), which they could've achieved simply by using an unsigned int (2^16). It's ultimately not saving any space.
    – canisrufus
    Commented Jan 19, 2012 at 20:56
  • When you refer to "memory usage" it's hard to tell whether you mean RAM or disk. In the early 90's, a 2 GB drive and 16-32 MB RAM were pretty high-end: saving some file space (or network bandwidth) would still be of importance. A responsible software engineer would want to think carefully through the implications for their future customers of time-space tradeoffs in their choices; in hindsight I would give them the benefit of the doubt unless the choice was obviously, devastatingly inefficient.
    – whuber
    Commented Jan 23, 2012 at 19:40
5

This is my take on it.

Shapefile format most likely evolved from ARC/INFO which had history dating back from its FORTRAN / PR1ME origins. All the ARC/INFO formats had this 100 byte header and the Big endianess of the File Code and File Length (e.g. Coverages, TINs).

When Shapefiles were made for ArcView 1, ESRI was focused on breaking into the Microsoft Windows market and the remainder of the Shapefile format is heavily focused on being little endian of PCs.

The constant switching between endianess was, presumeably the need to support the legacy origins whilst anticipating benefits on breaking into the platform.

Improve this answer
5
  • This sounds plausible. Thank you for the insight!
    – whuber
    Commented Jan 19, 2012 at 14:42
  • This is my favorite conjecture about the endianness. Now all we need is Dangermond to publish "The ESRI Tell All, Technical Edition" to see if you're right!
    – canisrufus
    Commented Jan 19, 2012 at 20:57
  • 2
    If the shapefile format evolved from ARC/INFO formats, it was considerably earlier than v7. In 1994 when I started at ESRI, AV2 was already out, and development work for ARC/INFO 7 was underway.
    – mkennedy
    Commented Jan 20, 2012 at 15:30
  • Good point, Melita. The crux of this reply--that some format choices might ultimately have Fortran origins--would still be true all the way back to the original Arc and Info applications.
    – whuber
    Commented Jan 23, 2012 at 19:36
  • Thanks @mkennedy, I removed reference to v7. I still remember the days that the original ARC/INFO user manuals (v3 .. v6 era) had headers which I believe was taken from FORTRAN code.
    – Stephen Quan
    Commented Jan 23, 2012 at 21:40
4

I always assumed that the endian split was caused by having two teams one on Sun Workstations and the other on PC's and them not meeting up until near the end of the development process.

I'd love to know what really happened.

Improve this answer
1
  • 3
    I think ESRI was a little more coordinated than that. Indeed, if anything, their software has a tendency to look like there has been too much committee involvement in its design.
    – whuber
    Commented Jan 18, 2012 at 16:35
0

I think somewhere back there I heard something about the dbf/foxpro origination.
That could have just been a weird dream I had though.

Improve this answer
1
  • 5
    The .shp and .shx parts, which are in question here, were designed completely independently of the .dbf format, which had been around for almost 20 years previously.
    – whuber
    Commented Jan 18, 2012 at 16:30
0

You have to understand shapefiles were introduced some 20 years ago, at that time there was a myriad of inconsistent and poorly designed file formats, so shapefiles are no exception. I've written a shapefile parser myself and I have to say I've had many more problems with parsing the DBF format compared to the shapefiles (.SHP) themselves.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.