2

I am trying to mosaic seven dem files together by using "mosaic to new raster". The largest one is about 100 GB, and the other six files are total 50 GB. I used "FIRST" mosaic method and the cell size is the same with the largest dem. And the boundary after mosaicing will not change, which is also the same with the largest one. So basiclly the output file should be about 150 GB or 200 GB something like that.

The mosaic work has taken a really long time, which is about 16 hrs and haven't finished yet. I checked the output folder and find there was only two files has been created, total 1.13 TB, which is unreasonable, because it is much larger than I expect. I am thinking if there is some temporary file there caused this and if the long time is reasonable for 100 GB DEM mosaic with 50 GB DEM?

  • Are you using ArcGIS? What version? What format are the dem files? Could you create a Mosaic Dataset instead? Maybe try the process with smaller files as a test. – Mattropolis Oct 5 '16 at 18:29
  • Usually in such case the input files are compressed and output is not which can mean 10 times bigger file than expected. – user30184 Oct 5 '16 at 19:40
  • 1
    Even if the output is compressed, I expect any temporary files will not be. And you won't be able to hold files that size in physical memory so I expect a lot of disk caching. Try mosaicking a couple of the smaller files first - just to confirm that the process is working and produces a valid result. If it does, try extrapolate the time taken to estimate time for the full dataset. – Mark Ireland Oct 5 '16 at 20:04
  • Thanks for your help! Yes, I am using ArcGIS 10.3, The format of DEM is tif.The process worked before for smaller DEM mosaic. First time to do this for such a big one. I think next step I will try to mosaic the one by one, instead of mosaic them together at one time. Thank you guys! – Yifan Wang Oct 5 '16 at 20:37
  • Please edit the question in response to requests for clarification. It is not fair to the volunteers who would answer your question to make them sift for critical information in comments. Generally, output is much larger than input when you request it (intentionally or otherwise). – Vince Oct 6 '16 at 1:51
1

As @MarkIreland commented:

Even if the output is compressed, I expect any temporary files will not be. And you won't be able to hold files that size in physical memory so I expect a lot of disk caching. Try mosaicking a couple of the smaller files first - just to confirm that the process is working and produces a valid result. If it does, try extrapolate the time taken to estimate time for the full dataset.

and @AlecZ commented:

This is true for all manner of geoprocessing, especially with raster DEMs - temporary files can grow substantially until they are written to a (compressed) output file. When that happens, the last step before completion, the temp files are deleted. If the process fails, the temp files will remain in state. I’ve found it necessary to change my temp directory (for Windows Environment) to a large external drive, when using large elevation datasets in Arc[GIS Desktop].

  • This is true for all manner of geoprocessing, especially with raster DEMs - temporary files can grow substantially until they are written to a (compressed) output file. When that happens, the last step before completion, the temp files are deleted. If the process fails, the temp files will remain in state. I’ve found it necessary to change my temp directory (for Windows Environment) to a large external drive, when using large elevation datasets in Arc. – AlecZ Feb 18 '18 at 3:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.