18

I would like to speed up the process corresponding to this command:

gdalwarp -t_srs EPSG:4326 -overwrite input.ntf output.tif

Is there a way to use parallel processing in GDAL? Semi-offtopic: If not, do you recommend a non-GDAL solution to speeding up a gdalwarp-like process?

Here are the web pages I've looked at:

Edit: I asked this question because I thought I saw my CPU% stay below 100% when processing gdalwarp. However, on a second look, it reached 555.5%.

Improve this question
2
  • 4
    Did you read about -multi in the document gdal.org/gdalwarp.html?
    – user30184
    Commented May 3, 2017 at 21:00
  • -multi made it even faster. The max CPU% I saw was 700% instead of 555%. The process took 22 seconds instead of 32 seconds.
    – Matt Kleinsmith
    Commented May 4, 2017 at 1:44

1 Answer 1

Reset to default
16

Yes, GDAL supports parallel processing, and this support applies to gdalwarp by default.

Use the -multi option with gdalwarp to enable multithreading, as opposed to only multiple cores.

Details:

Without -multi: 33.849s, and the CPU reached 555%. (multiple cores)

With -multi: 23.377s, and the CPU reached 700%. (multiple cores and multiple threads)

Raster size: 34721 x 20453, OS: Ubuntu 16.04, # Cores: 6, # Threads: 12

Credit: user30184 mentioned the -multi option in a comment.

Documentation: gdalwarp

Improve this answer
2
  • 4
    See the improved documentation (by trac.osgeo.org/gdal/changeset/38196) of gdalwarp -multi: Use multithreaded warping implementation. Two threads will be used to process chunks of image and perform input/output operation simultaneously. Note that computation is not multithreaded itself. To do that, you can use the -wo NUM_THREADS=val/ALL_CPUS option, which can be combined with -multi. See also gdal.org/…
    – user30184
    Commented May 7, 2017 at 9:24
  • Does this work with gdal_rasterize?
    – jzadra
    Commented Jan 5, 2021 at 2:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.