2

I have downloaded numerous hdf files from an ftp site. Half of them were level 1 and the rest were level 2. Lvl1 format is Afile.A1_AC.sample.hdf and the lvl 2 format is: Afile.A1_AC.sample.hdf.

How will I do a batch process in projecting all the level2 files only?

How will I filter these files and create a new directory?

Here's what I have so far:

def inputData(self, event):

    def findSDS (path, filter):
        for root, dirs, files in os.walk(path):
            for file in fnmatch.filter(files, filter):
                yield file


     for sds in findSDS('/path/to/file', '*.hdf'):
        inSDS = 'HDF4_SDS:' + '/path/to/file' + '/' + sds + ':01'
        path = 'D:/new/dir' 
        outTIFF = path + sds + '.tif' 
        cmd = 'gdalwarp -geoloc -t_srs "+proj=longlat +datum=WGS84" -te 113.205 1.120 157.105 2.005 %s %s' % (inSDS, outTIFF)

        os.system(cmd)
        os.mkdir(path)

(tried calling .bat from the python script and run the script. And it just created an empty/new directory(out_path). Tried calling the .bat from the cmd line and here's the output:

C:\sample>project.bat
C:\sample>set in_path=D:\path\to\file
C:\sample>set out_path=D:\newfolder
C:\sample>md D:\newfolder
A subdirectory or file D:\newfolder already exists.
C:\sample>cd D:\path\to\file
C:\sample>FORFILES /m *A1_AC*.hdf /C "cmd /c gdalwarp -geoloc -t_srs "+proj=longlat +datum=WGS84" -te 113.205 1.120 157.105 2.005 @path D:\newfolder@fname.tif"
ERROR: Invalid argument/option - '+datum=WGS84 -te 113.205 1.120 157.105 2.005 @path D:\newfolder@fname.tif'.
Type "FORFILES /?" for usage.

C:\sample>project.bat
C:\sample>set in_path=D:\path\to\file
C:\sample>set out_path=D:\newfolder
C:\sample>md D:\newfolder
C:\sample>cd D:\path\to\file
C:\sample>FORFILES /m *A1_AC*.hdf /C "cmd /c gdalwarp -geoloc -t_srs "+proj=longlat +datum=WGS84" -te 113.205 1.120 157.105 2.005 @path D:\newfolder@fname.tif"
ERROR: Files of type "*A1_AC*.hdf" not found.

C:\sample>projected.bat
C:\sample>set in_path=D:\path\to\file
C:\sample>set out_path=D:\newfolder
C:\sample>md D:\newfolder
C:\sample>cd /d D:\path\to\file
C:\sample>FORFILES /m *A1_AC*.hdf /C "cmd /c gdalwarp -geoloc -t_srs "+proj=longlat +datum=WGS84" -te 113.205 1.120 157.105 2.005 @path D:\newfolder/@fname.tif"
ERROR 1: Translating source or target SRS failed:
'+proj=longlat

Tried running the gdalwarp in the cmd line and it was successful:

C:\sample>gdalwarp -geoloc -t_srs "+proj=longlat +datum=WGS84" -te 113.205 1.120 157.105 2.005 HDF4_SDS:hdf:"Afile.A1_AC.sample.hdf":01 sample.tif
Creating output file that is 1941P x 1670L.
Processing input file HDF4_SDS:hdf:"Afile.A1_AC.sample.hdf":01.
0...10...20...30...40...50...60...70...80...90...100 - done.
2

Here is a solution as windows batch script, maybe it'll help:

set in_path=path_to_stored_hdfs
set out_path=path_to_proccessed_ones

md %out_path%

cd /d %in_path%

FORFILES /m *L2_LAC*.hdf /C "cmd /c gdalwarp -geoloc -t_srs EPSG:4326 -te 113.205 1.120 157.105 2.005 HDF4_SDS:hdf:@file:01 %out_path%\@fname.tif"

Where /m is a mask to match desired filenames - for further details: http://ss64.com/nt/forfiles.html

Looking at your code I understand you've set GDAL path in system so gdalwarp should work.

|improve this answer|||||
  • Hi, tried your script but the equal sign from the set in part is having a syntax error. What should I do? – user Oct 9 '14 at 3:18
  • Hi, of course "in" is reserved for scrippting - sorry for my mistake (tried to write an answer as pseudocode first and forgot to change it later). I've made an edit in answer, now you shouldn't get a syntax error. – jzol Oct 9 '14 at 4:50
  • I think I'm maybe doing wrong. Using the script I posted I just replaced everything from insds until os.mkdi(path) with your script. I've made an update in the python script. – user Oct 9 '14 at 6:13
  • the in_path is now causing the syntax error. I'm being dumb at this. – user Oct 9 '14 at 6:25
  • As I wrote in answer: script I posted will do the work - but it's windows script - you can save it as a *.bat and run it in windows (as an alternative way to your python script). Your solution looks like it needs some improvements: first - you're creating output dir (same) in every iteration, what's more first you run a command (wchich is suposed to save some data in output dir) a then you finnally create the output dir. Try also printing inSDS outTIFF - if they're correctly passed to gdalwarp command. – jzol Oct 9 '14 at 6:26
1

you can use batch hegtool, read this:

http://newsroom.gsfc.nasa.gov/sdptoolkit/HEG/HEGFAQ_CLI.html

and this:

http://newsroom.gsfc.nasa.gov/sdptoolkit/HEG/HEG_Batch_job_Help.htm

|improve this answer|||||

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.