Take the 2-minute tour ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.

Why does it take so long to copy a large raster dataset from one file geodatabase to another? I can copy the entire database in windows explorer in minutes but copying a single raster in ArcCatalog takes forever. I always assumed it is not advisable to use the windows explorer to copy data between to File Geodatabases but is there a safe way to do this? If so, how?

enter image description here

share|improve this question
    
Is it just a single raster (or a small handful) in the database? I've found that the amount of time to copy or write a raster to a FGDB increases with the number of other features/rasters in the db. I sort of have my own cutoff of 75-100 before I feel like it's too much slower (but that's from a programming standpoint too. Your mileage may vary). –  nicksan May 18 '12 at 19:54
    
usually I copy one or two datasets (size varies 1GB - 20GB). There about 50 other rasters in this FGDB. –  Jakub May 19 '12 at 2:18

1 Answer 1

up vote 8 down vote accepted

I think the key is the difference between how Windows Explorer treats a single file, and how ArcCatalog/ArcGIS treat a raster. There are a few factors here.

  1. In the File GDB format, a raster is not represented by a single file. It is made up of the data table, along with probably pyramids and indexes. Why this is important is because when ArcCatalog is copying the raster, it is first combining all of these separate files into what we see as the raster. It is then running it through the ArcGIS programming layer to copy it to a new File GDB. This is much more complicated on the surface than just using the Windows System resources to copy a file because it is an additional layer of programming.

  2. The next part is related to something that you see when you copy another feature class in ArcCatalog. If you copy and paste in ArcCatalog, it brings up a progress window showing the original and target featureclasses, and then an iterator that counts the number of features that have been transferred. This shows that it is iterating row by row through the featureclass. This makes sense from a data integrity standpoint as each feature is copied and verified before moving to the next. Where this is relevant for the Raster is that a raster is made up of hundreds, if not thousands of Points. If a raster dataset is copied the same way as a vector featureclass, then ArcCatalog is essentially iterating through each coordinate pair in the Raster, to copy it from one to another. Progress bar showing number of features copied in ArcCatalog

  3. The last factor is the size of the dataset. In your case, the File GDB you are showing has a table that is 20,671,104 KB, which is ~20GB. That is a large Raster, and definitely contains many millions of points. That is a lot of coordinate pairs to iterate through.

When you combine the size of your Raster dataset, with the need to iterate over each row (Coordinate pair), and run that all through the ArcGIS programming layer, you end up with a much more complex, and thus longer operation than copying a single file using the Windows System functions.

-------Edit

To answer your original question of a faster way to copy rasters between file geodatabases, I would say one thing to try would be writing a python script calling arcpy. Since you would then be directly interacting with the API, without the additional ArcCatalog/ArcMap code running, it may be slightly faster. A code sample follows:

import arcpy
from arcpy import env

env.workspace = "C:/data"
arcpy.Copy_management("olddata.gdb/raster1", "newdata.gdb/raster1")

I'm not sure if the speed difference would be notable, but it is an option.
The other option, as @blah238 referred to, is to not store rasters in a file geodatabase, but in a file based format, like "geotiff", or even an ArcINFO Grid, as the file can then be completely manipulated through the Windows interface.

Unfortunately, other than this, your options for working with the file geodatabase outside of the ArcGIS interface are limited. There is an API for the File GDB which allows programmers to interface with it in different software. One example currently is through the GDAL/OGR libraries, and soon, Quantum GIS. The issue with the API is that it Does Not support Raster datasets at this time. If this is something you would like to advocate for, I would add your support to this entry at the ArcGIS Ideas site.

share|improve this answer
3  
In other words, use file-based or DBMS-specific raster (such as Oracle Spatial GeoRaster) data types where possible. –  blah238 May 18 '12 at 22:17
    
Thanks. Yes some of my rasters are large. So there is no way to use the explorer or some other 3rd party utility? –  Jakub May 19 '12 at 2:29
2  
You could do it using explorer by duplicating the FGDB that contains the raster in explorer, then use Catalog to remove everything you don't want and finally use catalog to add everything else you do want (i.e. explorer does the heavy lifting and Catalog tidies up afterwards). This is a nasty hack and presumes that the other contents of the FGDB are small and take less time to kill/copy than your raster. –  MappaGnosis May 19 '12 at 12:52

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.