0

I try to use gdal_calc.py to scale and transform a time-series of MODIS images. I've done that when i worked with a single image in the -A variable and a mask in -B, but i need to work with a directory of 350 .tif images and convert to ENVI .bin raster. My problem is how to assign the -A variable to realize the operation with all the images and obtain 1 output file per image

To work with a single image i use:

gdal_calc.py -A image.tif -B MASK10000 --calc="A/B" --NoDataValue=0 --format=ENVI --outfile=image.bin

I use the linux bash, but maybe a python scrip in linux runs better

  • I would use Excel to create a batch file.. get a directory list of all the in images then enter/fill the first and last part, save as CSV, open in Notepad (or similar) and replace all commas with spaces - save as .bat and double click. Did you specifically want to do this in python using os.walk? – Michael Stimson Jun 1 '15 at 23:40
1

I use a simple way to solve this using the UNIX bash.

I made a script in the same path of images and ran it ("script.sh").

#!/bin/bash
for i in *.tif
do
gdal_calc.py -A $i -B MASK.dat --calc="A/B" --NoDataValue=0 --format=ENVI --outfile=directory/$i.bin
done
  • MASK.dat is a binary image with only 10,000 values to be scaled within the NDVI range.
  • "i" are all the MODIS images that I need to convert. "directory" is the path where all the outputs will be created.

This was to then use the image files in TIMESAT software.

| improve this answer | |
  • I'm in windows... but that reads sort of like DOS for %%A in (.tif) do* which works (very well); kids on windows these days are scared of command prompt so I avoid batch solutions. That wouldn't help you anyway as bash and DOS are only similar not the same. I like it, and hopefully this simple solution will help future readers. +1 – Michael Stimson Jun 10 '15 at 22:16
  • @MichaelMiles-Stimson do you think it is possible in windows as I would like to do this and have a real world problem? something like for %A in ("C:\filename*.tiff") do gdal_calc.py ..... thanks, any pointers to get a simple OSGEO4w version of OSCAR_P great UNIX answer would help. Basically all tiffs in a folder just multiple would be a start. I can ask A Q if needed. – If you do not know- just GIS Jul 8 '16 at 4:28
  • @Ifyoudonotknow-justGIS have a look at robvanderwoude.com/for.php and ss64.com/nt/for2.html about for loops, not very specific but yes you can (and I have done) a simple batch file but trust me when I say it's easier and more scope by using python to write a batch file and calling it with subprocess.Popen, have a look at gis.stackexchange.com/questions/191045/… if you ask a new Q be careful how you word it, it could be off topic asking for help about DOS. – Michael Stimson Jul 8 '16 at 5:08
0

There are two python ways to do this simply, use os.walk() to find the files and then either of create a batch file or use subprocess.Popen() to execute the command.

This code contains both.. default is Batch File which can be switched by setting UsingBat = False:

import os, sys, subprocess

BaseInFolder  = sys.argv[1]
BaseOutFolder = sys.argv[2]
MaskImage     = sys.argv[3]
UsingBat      = True

if UsingBat:
    FirstPartOfBat  = "GDAL_CALC.py -A "
    SecondPartOfBat = " -B " + MaskImage + " --calc=\"A/B\" --NoDataValue=0 --format=ENVI "#--outfile=image.bin
    TempDir = os.environ.get("Temp") # get the 'temp' directory
    BatFile = os.path.join(TempDir,"RunMe.bat")# use your own batch file if you like

    with open(BatFile,'w') as BatchWriter:
        for (path, dirs, files) in os.walk(BaseInFolder):
            for file in files:
                fName,fExt = os.path.splitext(file)
                # you might want to think of something here to limit the file types
                # like if len( fExt ) == 0: I don't have experience with MODIS so I can't say
                OutFileName = fName + fExt
                # if that name is already in use try another name
                if os.path.exists(os.path.join(BaseOutFolder,OutFileName)):
                    Try = 1
                    while os.path.exists(os.path.join(BaseOutFolder,OutFileName)):
                        OutFileName = fName + str(Try) + fExt
                        Try += 1

                # assemble the command line and write to batch file
                pBat = FirstPartOfBat + os.path.join(BaseOutFolder,file) + SecondPartOfBat + "--outfile=" +os.path.join(BaseOutFolder,OutFileName)
                BatchWriter.write(pBat + "\n") # can't forget newline.. Linux/Mac may be different
else:
    FirstPartOfCommand = ["GDAL_CALC.py","-A"]
    SecondPartOfCommand = ["-B",MaskImage,"--calc=\"A/B\"","--NoDataValue=0","--format=ENVI"]#--outfile=image.bin
    for (path, dirs, files) in os.walk(BaseInFolder):
        for file in files:
            fName,fExt = os.path.splitext(file)
            # you might want to think of something here to limit the file types
            # like if len( fExt ) == 0: I don't have experience with MODIS so I can't say
            OutFileName = fName + fExt
            # if that name is already in use try another name
            if os.path.exists(os.path.join(BaseOutFolder,OutFileName)):
                Try = 1
                while os.path.exists(os.path.join(BaseOutFolder,OutFileName)):
                    OutFileName = fName + str(Try) + fExt
                    Try += 1
            # got to use a new list, otherwise direct assingment modifies the existing list
            # see http://stackoverflow.com/questions/2612802/how-to-clone-or-copy-a-list-in-python
            pOpenCommand = list(FirstPartOfCommand)
            pOpenCommand.append(os.path.join(path,file))
            pOpenCommand.extend(SecondPartOfCommand)
            pOpenCommand.append("--outfile=" +os.path.join(BaseOutFolder,OutFileName))

            ThisProc = subprocess.Popen(pOpenCommand)
            ThisProc.wait() # wait until finished, you don't want 350 of them running at the same time

Using a batch file you need to compile strings and insert values where appropriate, write to file and then run the file (double click); Using Popen you need to create a list of each element of the command - and not forgetting to use .wait() on the process otherwise the code will spawn a process and move on to spawn another process before the previous one has exited - you will end up filling up your memory in a few seconds.

I haven't had anything to do with MODIS data so I can't help limiting the input files so you should think about how you are going to limit the files.. os.walk() returns every file in the current folder and all subfolders - you will need a way to detect if the file is suitable before trying to convert it.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.