Take the 2-minute tour ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.

I'm trying to convert a 20 GB LZW compressed geotiff to a tilepyramid using gdal2tiles. Unfortunately, gdal2tiles seems not to be able to handle this, after running it for 10 hours it still hasn't reached the first "dot" yet. I tried this with a smaller (1GB) file and it worked fine. My question now is, is there any other software that allows me to do this (commercial or open source)? One can create tiles from GlobalMapper, but I cannot load the 20GB file into memory.

share|improve this question
add comment

4 Answers 4

I had the same problem just a few weeks ago. For once, you use the option

--config GDAL_CACHEMAX 2047

(must stay below 2048 for the time being!) And second, you can copy the gdal2tiles.py script to another location and alter the for-loop which iterates over the tiles to count backward. in my version of gdal2tiles that’s at line 1174:

for ty in range(tmaxy, tminy-1, -1):

change to:

for ty in range(tminy, tmaxy+1):

then use both scripts with the same arguments at the same time and add -e (i.e. continue, don’t delete already existing files) Note, that you have to quit one of the scripts as they come towards 50% completion (of the base tiles), or you’ll get broken tiles. If you forget, just search for all files with size 0, delete them and start all over - with the -e switch ;)

hope that helped

share|improve this answer
    
A question: why must GDAL_CACHEMAX stay below 2047? I use it happily with 8GB or more for gdalwarp and the like... –  markusN Dec 12 '11 at 20:58
add comment

If you are using a computer with multicore processor you can find Parallel GDAL2Tiles (ppgdal2tiles) useful. In this kind of computers ppgdal2tiles should run faster than the "single-core" version. Quoting ppgdal2tiles web page:

"The standard version of GDAL2Tiles is using only one processing core at a time.

In case you have a computer with more processors (or one of the Mutli-Core processors from Intel or AMD) then several processes can be started to use the full computing power of your computer to finish the tile rendering as fast as possible."

share|improve this answer
    
And you can certainly ask Petr for help with this issue and support the project if it is one. -> maptiler.com –  relet Oct 1 '10 at 13:49
add comment

I would hope that FME would be able to do that. It certainly can do the tiling - and I'm confident it can handle the large file sizes. The raster pyramiding part is pretty straightforward, see Raster Pyramiding example in the samples and demos pages.

You can get an evaluation copy to try it on first

Mark Ireland, Product Evangelist, Safe Software

share|improve this answer
    
this is a borderline advertisement. To push it off the fence into a solid answer please summarize how one would use FME to produce tiles and link to documentation that covers the OPs issue directly rather than just the front page. Thanks. –  matt wilkie Sep 8 '11 at 22:09
    
Sorry. Wasn't meant to be. The raster pyramiding part is pretty straightforward (fmepedia.safe.com/articles/Samples_and_Demos/…). I was just a little vague as the question was more on performance and I didn't want to commit when I hadn't tried it personally. That's why I invited the OP to try it directly. –  Mark Ireland Dec 12 '11 at 22:28
add comment

Try the second patch from this issue for a parallel version of gdal2tiles.py using the 'multiprocessing' module: http://trac.osgeo.org/gdal/ticket/4379

I got a 5.5 x speedup on a 6 core CPU.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.