11

I've been able to accumulate a large free dropbox account and use it to Sync my GIS databases across several different PC's. I've just started doing this and havent had any problems thus far. But I am getting some red flags of warning that are scaring me.

There seems to be several duplicate files in the GeoDatabases. Usually something like:

a00000017.gdbtable (GIS-HP duplicate Copy)

I can manually delete these files with no problem, but I have no idea why they are popping up.

Also, instead of removing file locks, Dropbox keeps saying it is renaming them. So when I release one lock and gain another, it renames the file instead of deleting it and creating a new one. Although it still does delete the lock file when I have fully disconnected from the GeoDatabase.

Like I said, I haven't had any issues so far, but I am worried about long term data corruption while using Dropbox.

Has anyone used Dropbox and found problems using it to sync GIS data?

|improve this question
  • 3
    I don't have a definitive answer, but I'd suggest that you're wise to be cautious. I don't believe that DropBox is intended for accessing a complex data format like a file geodatabase, which consists of multiple discrete files, on multiple PCs simultaneously. This post shows some potential issues to be aware of – Stephen Lead Aug 21 '13 at 15:18
  • @StephenLead I've been backing up the Dropbox data daily. I used Amazon Cloud Drive for about a year with no issues, but that is so slow for syncing. Dropbox works way quicker and simpler. I really hope that it can handle it. – Cody Brown Aug 21 '13 at 15:28
  • 2
    It is better to compress the file geodatabase before it is used to share resources.arcgis.com/en/help/main/10.2/index.html#//… it is not designed to be edited with more than one user at one time - lck files should prevent this. – Mapperz♦ Aug 21 '13 at 16:31
  • Solid idea @Mapperz – Cody Brown Aug 21 '13 at 16:42
9

ESRI intentionally obfuscated the FileGDB file structure so people wouldn't try and mess with it internally:

A file geodatabase is stored as a folder of files. When you view the folder with Windows Explorer, apart from the .gdb extension, it looks like any other folder and you can view its contents. The folder contains cryptically named files that hold geographic data, attribute data, index files, .lock files, and a signature file, as well as other files. Each feature class or table in the geodatabase is stored in two or more files. By design, it is difficult, if not impossible, to tell which files make up any given dataset.

ESRI recommends that you copy FileGDB's with their tools and not your OS tools. However, it is possible to copy the whole database or rename it:

Although not recommended, it is possible to copy a file geodatabase to another location by copying the folder to another location (if you are using a Windows Vista operating system, you also need to change the extension of the new folder to .gdb; otherwise, ArcGIS cannot recognize it as a file geodatabase). Only copy a file geodatabase with Windows Explorer when you are certain there are no other processes accessing the data. The problem is that it can be difficult to tell whether other processes are accessing the data. You might think there are no other processes; however, a process on your computer or another one could be accessing it. If you happen to copy the folder when another process is accessing the data, the data may not copy properly. An improper copy operation is often not apparent—you might not notice any problems until you access a certain part of the new copy a few days later.

I would be wary of using dropbox to sync the files individually. The database structure just isn't built to be manipulated in that way. If you dataset is small enough, it may be a good application for a personal geodatabase or a sqlite database (though I haven't seen it confirmed that ArcGIS can write sqlite) if you have simple geometry.

There is a tool that will recover corrupted geodatabase in the event they are damaged:

|improve this answer
  • 2
    You quote: 'you might not notice any problems until you access a certain part of the new copy a few days later' scares the crap out of me. It would probably be way smarter to use shapefiles instead of a GeoDatabase. – Cody Brown Aug 21 '13 at 15:33
  • I have been using SkyDrive to sync file gdbs without issue for almost a year now. When we have proxy issues which sometimes means I can't access Microsoft services, then I normally close my SkyDrive and only allow it to sync again once access has been restored. I have edited toolboxes with models and scripts when my SkyDrive is open but not syncing properly, and I have not lost anything yet. – Cindy Jayakumar Aug 23 '13 at 6:08
  • 1
    FWIW I was editing a complex polygon layer in a file GDB so I disconnected Dropbox while I was working on it. I reconnected Dropbox after editing, so the files were backed up. 24 hours later, all my edits are gone, and (since I had disconnected dropbox during editing) there are no intermediate files to revert to. So it looks like Dropbox may sometimes corrupt a file geodatabase. – Stephen Lead May 14 '14 at 10:28
8

If you need to have multiple PCs accessing files, it would seem to make more sense to migrate to a true spatial server setup rather than an ad hoc one using Dropbox. This could be either a hosted server running ArcGIS or a PostGIS database running on a cloud web service like AWS. My preference would be for the latter. It'll take a little more work to set up, but you can define your own relations and scale up to as many computers as you need (depending on how you configure AWS), all for much less than you'd be paying Esri.

There are several SO posts that discuss PostGIS and ArcGIS here and here. The second one seems to have more up-to-date information in the comments below the accepted answer.

Without something more robust handling the traffic from the different clients, you're probably going to be constantly dealing with data corruption issues. At the very least, if you moved to shapefiles, you should check out GeoGit. This would give you basic revision control so shapefiles edited on one computer can be merged with edits from another without being completely overwritten.

|improve this answer
  • Web servers are a great way to manage GIS Data across several PC's. In fact I agree, it's probably the best way. The issue with it is bandwidth and processing speed. Usually the internet speed offsite is high enough to work with the data. But, if I want to do some heavy analysis of the data, there is no way I want to be pulling it across the web. That's why I prefer a solution like Dropbox. I show up onsite and the GeoDatabases are all up to date and stored locally on the PC. Great way to work. – Cody Brown Aug 22 '13 at 20:10
  • 3
    @codybrown But you're still dealing with pushing data over the Internet with Dropbox, not to mention the sync issues you're getting, probably because it's taking so long to update from one client to another. At least a server would be optimized for that task. If you're really just looking to have a storage solution for data while you work from home, work, or client site, then a large removal hard drive is the only way to avoid latency and potential sync conflicts, assuming you're the only one using the data at any given time. – Richard D Aug 22 '13 at 21:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.