PyQGIS to use Taudem

Question

I'm using QGIS 3.34 (integrated with Python 3.9) for Windows 10. I'm trying to use MPI to execute Taudem's PitRemove in python. For some reason, it runs perfectly fine in a separate python shell or in Windows command line but I haven't been able to get it working in PyQGIS. My code goes:

mpiexec_path = r"C:\Users\favendan\Desktop\MicrosoftMPI\Bin\mpiexec.exe"
pitremove_path = r"C:\Users\favendan\Desktop\TauDEM537exeWin64\PitRemove.exe"
D4 = r"C:\Users\favendan\Desktop\D4_resampled.tif"

mpi_command = [mpiexec_path, "-n", "8", pitremove_path, D4]
    
subprocess.run(mpi_command, shell=True, stdout=subprocess.PIPE)

The result of this is:

CompletedProcess(args=['C:\\Users\\favendan\\Desktop\\MicrosoftMPI\\Bin\\mpiexec.exe', '-n', '8', 'C:\\Users\\favendan\\Desktop\\TauDEM537exeWin64\\PitRemove.exe', 'C:\\Users\\favendan\\Desktop\\D4_resampled.tif'], returncode=3221225781, stdout=b'')

It doesn't indicate any particular errors (just 3221225781), and doesn't output anything. I have tried changing several things: directories with no spaces for all executables, checked and reinstalled all dependencies: GDAL, MPI, M. Visual C++ (https://hydrology.usu.edu/taudem/taudem5/downloads.html), including PitRemove.exe to the firewall exception. I have also tried several ways to run this: subprocess.call(); subprocess.Popen(), os.system(). None of this seems to work.

I have tested MPI in PyQGIS with other processes (such as opening python scripts), and it works fine, so I know the problem is not MPI. It just doesn't like Taudem.

Unfortunately, the plugin that seemed to work to add Taudem to the processing toolbox of QGIS is down since a year or so and I haven't been able to find another one.

Anyone knows what the problem could be or what can I check for this problem?

Answer 1

The solution for me was to declare an environment in subprocess.run:

env = os.environ.copy()
env["GDAL_DATA"] = "C:\\Program Files\\GDAL\\gdal-data"
env["PATH"] = (
            "C:\\GDAL"
            + os.pathsep
            + "C:\\Program Files\\GDAL"
            + os.pathsep
            + "C:\\Program Files\\TauDEM\\TauDEM5Exe")

Then either

subprocess.run([#mpiexecdir,"-n", "#ncores",#PitRemovedir, #DEMdir],shell=True,env=env,stdout=subprocess.PIPE,stderr=subprocess.PIPE,creationflags=subprocess.CREATE_NO_WINDOW)

or

subprocess.run([#PitRemovedir, #DEMdir],shell=True,env=env,stdout=subprocess.PIPE,stderr=subprocess.PIPE,creationflags=subprocess.CREATE_NO_WINDOW)

works well.

I had come across some similar solutions, but these did not work for me:

• https://stackoverflow.com/questions/33730231/qgis-python-script-returning-non-zero-exit-status-1
• Problem running shell command with PyQGIS

Furthermore, there is another plugin for QGIS 3 called "drainage" which can be handy. It performs all TauDEM processing as batch.