11

I would like to speed up the process corresponding to this command:

gdalwarp -t_srs EPSG:4326 -overwrite input.ntf output.tif

Is there a way to use parallel processing in GDAL? Semi-offtopic: If not, do you recommend a non-GDAL solution to speeding up a gdalwarp-like process?

Here are the web pages I've looked at:

Edit: I asked this question because I thought I saw my CPU% stay below 100% when processing gdalwarp. However, on a second look, it reached 555.5%.

  • 3
    Did you read about -multi in the document gdal.org/gdalwarp.html? – user30184 May 3 '17 at 21:00
  • -multi made it even faster. The max CPU% I saw was 700% instead of 555%. The process took 22 seconds instead of 32 seconds. – Matt Kleinsmith May 4 '17 at 1:44
10

Yes, GDAL supports parallel processing, and this support applies to gdalwarp by default.

Use the -multi option with gdalwarp to enable multithreading, as opposed to only multiple cores.

Details:

Without -multi: 33.849s, and the CPU reached 555%. (multiple cores)

With -multi: 23.377s, and the CPU reached 700%. (multiple cores and multiple threads)

Raster size: 34721 x 20453, OS: Ubuntu 16.04, # Cores: 6, # Threads: 12


Credit: user30184 mentioned the -multi option in a comment.

Documentation: gdalwarp

  • 2
    See the improved documentation (by trac.osgeo.org/gdal/changeset/38196) of gdalwarp -multi: Use multithreaded warping implementation. Two threads will be used to process chunks of image and perform input/output operation simultaneously. Note that computation is not multithreaded itself. To do that, you can use the -wo NUM_THREADS=val/ALL_CPUS option, which can be combined with -multi. See also gdal.org/… – user30184 May 7 '17 at 9:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.