3

I am building an iterator that can go through several levels of folders and subfolders and perform some calculations on the entirety of data. It is supposed to be capable of going through folders set up by years, months and days. In folders for days would be hourly data (images). Everything seems to work in an earlier version I received from a member of this community (GDAL program produces only zeroes, regardless), but that one is only capable of going through one folder of data. I added an iterator that would go through the folder structure, that seems to cause GDAls function GetRasterBand() to not work, for reasons I do not understand. Particularly, as it seems to work just a few lines earlier in the script on the same type of file.

Here is the code:

def circle1():
    path = Path(path_top)

    ye = path_refer[:4]
    mo = path_refer[4:6]
    da = path_refer[6:8]

    if ye.startswith("*") and ye != "***":
        ye = "*"
        mo = path_refer[1:3]
        da = path_refer[3:5]
    elif mo.startswith("*") and mo != "**":
        mo = "*"
        da = path_refer[5:7]
    elif path_refer == "***":
        ye = "*"
        mo = "*"
        da = "*"

    arr = numpy.zeros((rows, columns), dtype=np_band_type)

    for s in path.glob('{}/{}/{}'.format(ye, mo, da)):
        if os.path.isdir(s):
            listdir = os.listdir(s)
            os.chdir(os.path.join(path_top, s))

            for a_img in listdir:
                if a_img.startswith(refer[:5]) and int(a_img[5:9]) >= year_beg and int(a_img[5:9]) <= year_end:
                    for b_img in listdir:
                        if b_img.startswith(refer[:5]) and int(b_img[9:11]) >= month_beg and int(b_img[9:11]) <= month_end:
                            for c_img in listdir:
                                if c_img.startswith(refer[:5]) and int(c_img[11:13]) >= day_beg and int(c_img[11:13]) <= day_end:
                                    for d_img in listdir:
                                        if d_img.startswith(refer[:5]) and int(d_img[-8:-6]) >= hour_beg and int(d_img[-8:-6]) <= hour_end:
                                            ds = gdal.Open(str(d_img), gdal.GA_ReadOnly)

                                            band = ds.GetRasterBand(b_num_wor)
                                            b_arr = band.ReadAsArray(0, 0, columns, rows).astype(np_band_type)
                                            for r in range(rows):
                                                for c in range(columns):
                                                    if param_type == "lt":
                                                        if param_x < b_arr[r,c]:
                                                            arr[r,c] += 1
                                                    elif param_type == "let":
                                                        if param_x > b_arr[r,c]:
                                                            arr[r,c] += 1
                                                    elif param_type == "dr":
                                                        if param_x < b_arr[r,c] < param_y:
                                                            arr[r,c] += 1

    return arr

path_refer variable consists of yyyymmdd integer (like 20140101, for January 1st, 2014), rows and columns are extracted from a file earlier in the program, while np_band_type is also extracted earlier:

measure_source = gdal.Open(MyReferenceFile)
v_band = measure_source.GetRasterBand(b_num_wor)
gdal_band_type = v_band.DataType
np_band_type = gdal_array.GDALTypeCodeToNumericTypeCode(gdal_band_type)

The param_x and param_y as well as the b_num_wor (raster band number) are all extracted form a text file. And the v_band = ... line works, while the nearly identical line within the function band = ds.GetRasterBand(b_num_wor) return an error:

line 175, in circle1
    band = ds.GetRasterBand(b_num_wor)
AttributeError: 'NoneType' object has no attribute 'GetRasterBand' 

I have looked at some other answers (Did GDAL install correctly in Winpython?, GetRasterBand() method for gdal in Python) but none seem to work for me. There is likely a better way to do the iteration, though I am currently more interested in why the band reading part doesn't work.

  • If you do >>> print type(ds) right after your gdal.Open, what does it say? If it says >>> <type 'NoneType'>, print your d_img variable. Is it forming a correct path to the image from your cwd? – Roberto Ribeiro Jan 23 '18 at 14:36
6

To debug such issue, start with the error:

line 175, in circle1   
    band = ds.GetRasterBand(b_num_wor)  
AttributeError: 'NoneType' object has no attribute 'GetRasterBand'

This means that ds is null, so you can't do anything with it.

Go up your code where ds is set

 ds = gdal.Open(str(d_img), gdal.GA_ReadOnly)

It use d_img, which is expected to be the image name. Your variable likely contains the wrong name, you can try to print it out to validate it.

What is d_img? Keep moving up, it is a member of listdir that is created by

listdir = os.listdir(s)

The doc says it returns the filename in the s directory. So you are trying to open this file, but you only have the filename, not its path. Oh wait, you also did os.chdir(os.path.join(path_top, s)) in an attempt to set the "base" directory and open the file from there. Though there is a path_top component that is now added to the mix, and it's much likely creating troubles.

So, now that the source of is issue is identified, you can easily fix it. Since your got the filename from the s directory, just open it from there. Don't use relative path but build an absolute one (always... it will save you lots of troubles down the line)

 ds = gdal.Open(os.path.join(s,d_img), gdal.GA_ReadOnly)
  • I tried debuging it the way you suggested, but first printed ds.GetGeoTransform and it prints it(-204372.81516064578, 1000.0, 0.0, 144649.46126293932, 0.0, -1000.0) but then returns an Attribute Error . So I added the os.path.join(s, d_img) to the ds = gdal.Open() function and I still get the Attribute Error. I also used print(type(ds)) as suggested by @Roberto Ribeiro and that returned: <class 'osgeo.gdal.Dataset'> <class 'NoneType'>. As it seems the problem is somwhere around s, wich should represent folders but it represents files as the last part of path.glob() are day folders – peroman200 Jan 23 '18 at 15:04
  • 1
    PS: May have been solved, by your correction as well as also removing .hdr files I had stored besides the files. They were paired by names and (obviously) don't have bands. As soon as they were removed, the program started working. Just hoping I didn't create an infinate loop. – peroman200 Jan 23 '18 at 15:22
  • Here's a thing about GDAL's Python bindings that isn't widely recognized: the bindings don't raise normal Python exceptions unless you specify that you want them. GDAL hides errors by default. If you begin your script with import gdal; gdal.UseExceptions() you will get a Python exception with a message that tells you why the file can't be opened instead of an opaque "AttributeError: 'NoneType' object has no attribute 'GetRasterBand'" error. – sgillies Jan 23 '18 at 18:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.