3

Many field sciences require data management plans for their research data that specify how the data will be submitted for archival storage. These data come from various devices and are often in proprietary formats that require processing. For example, Trimble GPS devices produce SSF format files but these are not suitable for longterm storage because they are proprietary and require a Trimble software license.

If you worked for a library or the US Forest Service what vector format would you consider the most sustainable for people to use when they contribute data to your archive?

While not an open format Shapefile is a defacto standard and widely readable (in practice this is what people are using). To be clear, these are not archives that are served online they're stored, perhaps burned onto archival DVD.

Relevant Links

  1. GDAL - Geospatial Data Abstraction Library
  2. Sustainability of Digital Formats: Planning for Library of Congress Collections
  3. OGC standards working group
  4. FOSS Open Standards/Comparison of File Formats
  • 1
    Didn't you just ask then delete this question? Even DVD isn't guaranteed to be suitable for long term storage, there was a time when tapes were considered the only long term storage solution and now I don't think you can even buy a tape drive and DVD media can degrade even stored in optimal conditions. – Michael Stimson Mar 31 '17 at 5:03
  • I notice that you did not edit your earlier question at gis.stackexchange.com/questions/234228/… and instead deleted and re-asked it. The latter is not prevented but the former is preferred. – PolyGeo Mar 31 '17 at 5:04
  • 1
    What type of vector data archive does this library or the US Forest Service maintain? If they use different spatial data storage formats for their archives then the answer may vary, which will make this too broad for focussed Q&A. – PolyGeo Mar 31 '17 at 5:05
  • The biggest problem I have seen with archives of data is not the format but with the bucket 'o data approach, after only a very short time it's hard to locate any data that you know you have or to understand the data you do find.. Index your data with a table (spreadsheet or database) and keep/populate metadata. Quite often multiple versions of the same data are archived but are slightly different, understanding which version is the correct one comes down to metadata - if it's not there you have a mess! – Michael Stimson Mar 31 '17 at 5:13
  • When improvement to a question is requested via comments the way to do that is by using the edit button to revise it. – PolyGeo Mar 31 '17 at 5:13
2

In my opinion, Shapefile is a good choice, but it has many limitations for historical reasons. For example, a single shapefile cannot be greater than 2GB, etc. You have to decide whether to commit to those restrictions if you use shapefile as the archival format. There is a good discussion at GIS.SE as to Are there any attempts to replace the shapefile?.

For archival purposes, a good choice may be to use text formats, such as GML, GeoJSON mentioned in the above post, or even Well Known Text (WKT). Text formats are most portable, readable forever, and (so far) without license issues. We can always reconstruct auxiliary files 50 years later from the basic data. To separate concerns, one can choose an efficient/reliable method/format to compress the text files before archiving them.

2

The shapefile could be considered an open format in that its technical description is published and routines for reading and writing the format can be created without requesting permission or paying license fees.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.