Take the 2-minute tour ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.

Back in the days when I was in the university I had a "Organization and tidiness" problem – I was unorganized and kept my layers in different folders without distinct names and hence had multiple copies of each layer.

Ever since I started working, I've improved a lot – I keep special folders with special subfolders. I name my layers according to a system which lets me be a bit more neat, but as I still have to manage multiple copies of layers (As Autocad and ArcGIS have their differences when dealing with non-Latin languages, I have to keep a copy adjusted for each program), I'd like to hear from your experiences and maybe learn a few tips from you:

  1. How do you organize your layers? How do name them? By name, date, contents, customer?
  2. How do you organize or deal with multiple copies (more acute: how do you update several copies at once)?

Note: I'm talking from the analyst/DBA POV and not from a web-developer's/web-manager's POV (I'm talking about organizing the layers for myself and maybe two more GIS workers, not more).

share|improve this question
5  
A good question. Actually it's not a qestion, it's a quest. A question leads to a single or narrow-ish set of answers, and once settled, it's over. A quest is an ongoing thing, an adventure that may never have a definitive end, and that's what you have here. Resign yourself to the truth that no matter what convention you settle on, it will not work completely or thouroughly. That said, there are conventions you can use to make the path smoother and easier travelled. Kevin's answer, and the follow on comments, are a very good start in this regard. –  matt wilkie Aug 30 '11 at 15:27
add comment

3 Answers

up vote 14 down vote accepted

This is a wicked problem. We've tried various systems, which have all worked to varying degree for a time, and eventually grown unwieldly and started to fall apart as more and edge cases which don't quite fit are encountered. That said, each of the systems we've used is way better than nothing, proving the maxim that any system is better than no system.

Here is a thumbnail overview of our current practice:

Put everything except rasters into a file geodatabase, the fewer the better. Don't nest feature classes under feature datasets unless they are related in some manner (e.g. hydro>streams, hydro>lakes, hydro>wetlands, etc.). This leads to a big long list at the top of the fgdb but that is an acceptable evil.

Create layer files for all the feature classes and organize that instead, this gives a lot of freedom to name as needed, using unsupported characters etc.*, and ability to move and rename as circumstances change. It also allows duplication without redundancy, for example one set of layers grouped according to nominal scale (50k, 250k...), another by region (AK, YT...) , a third by theme (caribou, land use, transportation...), and a fourth by client while the datastore itself remains unchanged.

For duplicates use shortcuts instead of the layer files themselves, otherwise there are too many things to update when things change. Configure ArcCatalog to show shortcuts: *Tools > Options > file types: .lnk (Limitations: preview & metadata don't work, you can't follow the shortcut to its source in ArcCatalog. This can be remedied using Symbolic Links instead of shortcuts, see Link Shell Extension)

*(tip: add the Layers folder as a Start Menu toolbar so they're always at your finger tips.)

Z:\Layers\
          Base\
          Thematic\
          Reference\
          All Dressed Base (250k).lyr
          Administration Boundaries (1000k).lyr
          ...
Z:\Raster\
          Landsat\
          Orthos\
Z:\Data\
        Foo_50k.gdb
        Foo_250k.gdb
        NoScale.gdb

Map compositions and outputs (print files, pdf's, exports, etc.) which by nature are more dynamic and variable are stored and organized differently somewhere else. This is the part which has been harder for us. We currently use a dedicated drive with folders named according to Job# (doing it again I'd use date instead, '2010-10-26') and sub folders for project specific data and results/deliberables. A spreadsheet index lists all the job numbers (folder name), their corresponding map titles and client. Ex:

W:\Foo_0123\
            Foobarmap_001.mxd
            Docs\
                 ReadMe.doc
            Data\
                 buffers_2000m.shp
                 gps_tracks.csv
            Output\
                   Foobarmap_001.pdf
            Deliverables

Keeping the index up to date is a friction point, people don't like to do it, avoid it, and are inconsistent with naming etc. (using a database instead of spreadsheet would help). Using a numerical folder name convention also makes it very difficult to the map for project X without the index, another notable source of friction. Ideally the index would be a clickable html page which is automatically generated from a db application. That is whole 'nother project though.

Key principles:

  • separate the slowly changing and often reused stuff from the dynamic and variable, and treat them differently
  • Don't duplicate unecessarily, use layer files and shortcuts/links wherever possible.
  • don't change systems too frequently, give each a solid try.

I very much welcome examples of other structures, as I said we're not content with what we have. :)

share|improve this answer
    
I lightly chastised someone yesterday for posting something too big and long, and here I go and do the same thing, just without the pictures. What do you think, is it better to present a cohesive whole or break things into modular pieces, which can each voted up/down on their own merit, but possibly break or hide their integration with the others? Talk about it on meta: Long and cohesive or short and modular –  matt wilkie Oct 27 '10 at 19:02
    
Wow. What a through filing system (I've read it four times already and not still sure I got it all). Two comments that stood out for me, as a bind AutoCAD and ArcGIS user: 1. how do I fit into this system the storage of DWG's? 2. GeoDatabase is a great way for keeping organized. The only problem I have is that AutoCAD map doesn't see GDB but only shapefiles. But thanks, I'll take tips from your thorough system... –  jonatr Oct 28 '10 at 20:55
    
keep in mind this system grew to be like this over the course of 15 years or so, and is tailored to how we work. There should be some portable elements though. As for interoperability with CAD, keep on ESRI's case to honour their commitment to publish an open API to the file geodatabase. –  matt wilkie Oct 30 '10 at 7:03
1  
Ditto on the feature datasets. It's sort of a useless feature except in ArcCatalog. They're supposed to group layers of common use and spatial reference, but a programmer never sees a feature dataset until it gets in the way of looping through the layers in a workspace. When using different projections, separate databases seem better than feature datasets. –  Tim Rourke Nov 2 '10 at 14:20
    
@Tim I believe Feature Datasets are the conceptual decendant of ArcInfo coverages, namely they're to be a means of grouping disparate geometry types that describe a common "thing" into a single basket. So you can have for example watercourses (lines), waterbodies (polygons), and rocks-in-water (points) together. I don't know why they're not presented more directly to the programmer. –  matt wilkie Nov 2 '10 at 15:47
show 1 more comment

If other people will be accessing the data in your system, you cannot make the organization schema meaningful for only yourself; you must keep their use of the system in mind. If you don't consider them, you'll be spending alot of time answering questions such as "where is the landuse data" and "why can't I find the [insert dataset here]?"

In maintaining such a system for many years, I found that people can't find data if it is first organized by source, e.g. c:\CensusBureau\Roads and c:\ESRI\Countries. Instead, I recommend to list the data thematically first, then by source in case you have multiple sources, e.g. c:\Roads\CensusBureau and c:\Roads\LocalGovt.

Likewise I wouldn't separate rasters and vectors into different directories. However, it may be necessary to split them onto different physical or logical drives if you have alot of raster data that won't fit onto one drive.

I recommend the following directory structure. Theme\SourceYear, where Theme is the thematic layer, Source is an abbreviated name for the data source, and Year is the year the data represents on the ground. In this scenario, TIGER Roads from the Census Bureau would be located in \Roads\Census00 and \Roads\Census10 (or replace 'Census' with 'TIGER').

Be aware that certain extensions in ArcGIS don't work with file names longer than 13 characters. I can't remember which extension, I just remember this being a problem.

share|improve this answer
    
Thanks Kevin, what about file name convention? I'm thinking a solution like <Object>_<Location>_<Range>_<Date>_<FileFormat>_<Resolution>.<ext> coast_eu_340509.76N0080201.23E_350509.76N_0090201.23E_2011_shp.zip box_eu_340509.76N0080201.23E_350509.76N_0090201.23E_2011_tiff.zip. Do you think it is a valid idea? –  Jade Aug 30 '11 at 12:55
3  
Those filenames could become very cumbersome to use on a command line or in a program. They would also lead to a very wide table of contents and/or legend in ArcMap (by default at least). I would opt for shorter file names, e.g. just object or object and date, and use standard metadata or at least a readme file to relay the rest of the information. Just my opinion. –  user3461 Aug 30 '11 at 13:36
3  
I agree with Kevin. My current company has a legacy file name convention (that I'm in the process of changing) that mandates loooong file names and it is very cumbersome for just the reasons kevin mentioned. Two additional thoughts 1) Much of what you have in file name could be broken out into folders and sorted in the file structure - not hte file name; 2) the multiple periods/dots (.) in the file name may cause issues accessing files through certain software and/or programatically. typically characters after a (.) are the file extension and not additional file name componenets. –  hgil Aug 30 '11 at 14:19
add comment

We work on a project level for cad files guess it depends on how your particular work flow is set up, we have our master working project then prepare any additional datastores from this in an export script at the end of the editing session.

datadir\cad\cadastre.dgn
datadir\srv\fuel.dgn
datadir\srv\sewerage.dgn
datadir\map\base.dgn
datadir\map\printsets.dgn
...

then each file has levels/layers/features named with an identifier
sewPipe
sewManhole
sewPit
...

We then export it all to SQL spatial instead of reading our working project files where it is displayed to the user via Mapguide or whatever flavour GIS app needed.

The GIS layers are sorted by feature name with identifiers and similar folder layout to allow sorting.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.