2

I am asking this question because I am at a loss and dont have a good enough understanding of all the underlying operations going on 'under the hood' of Windows/Arcgis.

  • What is the root cause of fgdb performance issues on network drives?

I have done some testing regarding performance of fgdb's vs shapefiles, network vs local drive.

Test results

B: drive is my local computer drive. K: drive is the local network drive for geodata.

Method: I use 5 point feature classes with random generated points, all within the same extent. The feature classes have respectively 100 000, 10 000, 1000, 100 and 10 points. The left side of the diagram shows time used while transferring higher number of points, while the right side shows time used while exporting feature classes with 10 points (all duplicates of the 10 point file). The Y-axis is time in seconds.

Note that all results on my local computer drive are pretty much flat compared to transfers on the network.

Monitoring the task manager on the network tab on my computer, i can see that there are large periods of network 'silence' followed by a short burst of activity (1 featureclass), then followed by anoth 40'ish seconds of nothing.I am asuming it is because of schema locks, But I would like to know what causes it and what can be done to prevent it.

Backround information:
I work within a very large government organization (~1300'ish ArgGis licenses) with several regional offices and a small'ish central IT group. We are still working in ArcGIS 9.3.1, but are going to upgrade to 10.3 in the fall. We are not a GIS-dedicated organisation, GIS is only a tool for supporting the decisionmakers here. Only my colleague and I are GIS-engineers in regional office (500 employees, 70 Arcgis licenses).
We have a central database (SDE on Windows Server 2008 R2 that contains most data on a national level) and a local (physical server) that is dedicated for geodata within each regional office. Certain datasets (large raster datasets, DEMs) are divided on a national level and also stored on the local server in the regional office.
The infrastructure is set up by IT. Geodata is only a small part of this, so the geodata considerations do not always take priority with the entirety of the IT system (i.e. turning off opportunistic locking is not an option).
Due to our large organization and legal requirements, working on the local computer drive should be avoided due to lack of backup. In theory, all geodatawork should be done on the geodata network drive (local in-house server).
When we upgrade to ArcGIS 10.3, with the easy option of 'let this be my home directory for this MXD' option, I would very much like to use the filegeodatabases to help keep all data organised, as there are several new MXD per week with plenty of scratch data floating around everywhere. However, with the performance issues that I have found, It feels we will still be stuck with shapefiles and a hard-to-keep-organized data environment.

  • I think that your question in its current form is a discussion that you might be better having with your local Esri support. I strongly recommend that you take our Tour so that our focussed Q&A format, which is very different to a discussion forum, becomes clearer. For it to be suitable here I recommend using the edit button to revise it so that it asks a single question; it is the "best practice" one that I would remove. Also, you seem to be blaming file geodatabases but every test you timed was to a file geodatabase - what happens when you export to a shapefile? – PolyGeo Jun 10 '15 at 8:18
  • Hey PolyGeo - I agree with your assessment, but I am getting a little desperate. ESRI no longer provides support for 9.3.1, so the standard answer is 'you should upgrade'. Regarding your question, exporting to shapefile, see the 4th test example - from fdgb to shapefile takes 60 seconds. – Kristian Herner Jun 10 '15 at 8:34
  • Sorry, I must have overlooked that last test result. Please be aware from our help center that "We are not a GIS Software Support Site" and also that we are manned by volunteers so the degree of help that can be offered really relies on how much you can focus your question: meta.gis.stackexchange.com/questions/3672/… – PolyGeo Jun 10 '15 at 8:44
  • 1
    Best practice is to avoid use of file geodatabases on network drives. This is a fundamental characteristic of FGDB. Nothing, including upgrade, is going to change this. There may be driver-specific settings to improve throughput, but there is nothing anyone here can do about it. As such, this isn't really even a question. – Vince Jun 10 '15 at 12:31
  • Thank you for your answer Vince, so I guess your recommendation would be to forego FGDB all together and only use shapefiles? We are not allowed to store data on local computers due to lack of backup. We generate a lot of small datasets that are usually stored with an MXD for creating a specific map, for example. We do this several times a week, among several people, so it is not practical for us to keep all data stored in a single SDE due to how messy it would get. Any suggestions are welcomed! – Kristian Herner Jul 1 '15 at 11:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.