Take the 2-minute tour ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.

My company has collected about 30 TBs of GIS data over the past 8 years, and I always find myself asking the following questions:

  1. What type of data do we have for a given geographic area?
  2. What are the details about that data (e.g., resolution in meters per pixel)?
  3. Where does the data exist on the hard drive so I can actually use it?
  4. Have we processed the data already, or is it in an unaltered form from the source?

Up to and including now, I have attempted to address these questions by devising an appropriate folder and file taxonomy/hierarchy. Does anyone have any ideas/suggestions on some understandable, perhaps even standard ways of organizing GIS data using files and folders?

I am also open to learning more about how using a database might benefit my company; we are software developers, not GIS experts, so I suspect we're quite a bit behind the curve on how best to approach the problem of storing/organizing GIS data for ease of use. I did see the question Best practices for managing geospatial data but was only able to draw marginal use out of the answers because I am so unfamiliar with geodatabases.

UPDATE: This last week I spent a fair bit of time reading about GIS databases, and got started familiarizing myself with PostGIS. Long-term, I think we will end up moving towards employment of a database plus metadata server as recommended by JasonBirch in Best practices for managing geospatial data.

share|improve this question
7  
Check out this question: gis.stackexchange.com/questions/2976/… –  Derek Swingley Nov 16 '10 at 23:37
    
Thanks, that question is definitely related and provides some good background info. –  Sipp Nov 17 '10 at 17:20

3 Answers 3

It seems to me that you need a set of metadata to store this information, and a retrieval system that uses the metadata to let you extract data on the basis of the information.

I think you'd want a solution that supported an OGC Catalog Service, for maximum interoperability. I've seen colleagues use Deegree - though of course there are other solutions you should check out.

Here's an example of how we tied Deegree into our software (the live demo is down for maintenance right now - wouldn't you know! - but should be back up next week)

As for file naming, if you have a catalog service and delivery mechanism, then there's less of an issue about what the files are named and where they are. Otherwise I think it depends on how you look for the data. Do you first start by narrowing down the geographic area, or the type of data? That will determine whether the hierarchy starts by splitting data into tiles, then types of data per tile; or by splitting it into types of data, each of which has a set of tiles.

Of course with a spatial database you don't have the same issues about dividing the data into tiles, so that is often a preferential method - providing the end use application supports using that type of data.

share|improve this answer
    
Thanks for the suggestions Mark. It seems you are suggesting that there are a few components at play here: the metadata itself (e.g., an XML file), a retrieval system (Deegree?) that knows how to find data based on certain metadata requriements from the user, and a storage backend component (e.g., PostGIS?) that stores both data and metadata. Is that accurate? –  Sipp Nov 20 '10 at 17:29

I would choose SpatiaLite which is an one-file database where you can insert all your shapefiles, rasters and tables. Then as a relational SQL database, you have the power of SQL queries in your disposal to do all necessary actions (join,select,merge,union,split etc) between attributes and files.

SpatiaLite is also accessible from programming languages such as Python for a greater degree of automation. The sky is the limit.

SpatiaLite Documentation and tutorials

share|improve this answer

If you are actually trying to edit data or develop a map, you will need to keep the data you are actively working on separate from the data you started with. When I start a project, I create a SourceData folder, with subdirectories named by the type of data (DEM, Orthophoto, Hydrology, etc.) This will hold all of the layers that I am merely using for reference. Any data I am working on will be copied into a different folder called Working. The Working folder holds data, MXDs and anything else that I modify or create in subdirectories that usually correlate to a phase of the project (MXDs, RoadEdits, Delivery, etc)

In addition to the actual GIS data, you should create a Communications or Specifications folder to hold any documents from your client/internal client/professor. This can serve as metadata when you come back to the project at a later date, as well as creating a centralized location where anyone else can see what is supposed to be happening.

share|improve this answer
    
Good points; our company makes maps that our software uses, and we have already developed a folder scheme for separating "raw" data from "working" data from "finalized" data. One of the problems is tracking down what set of raw data was used as the original basis for a final map; seems like your suggestion for a "Specifications" folder would address that. For each map we create, we would be sure to note what raw data source was used in the creation of the map (something we currently do not do). Thanks for the tips! –  Sipp Nov 20 '10 at 17:35

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.