Export numpy array as TIFF with rasterio fail with the folder name :No such file or directory

Question

I experienced unexpected behaviour of rasterio and I couldn't understand yet where is my mistake. I have opened rasterio TIFF, did some manipulations on it as numpy array and then export it as TIFF using rasterio.open and dst.write.
The problem is that the saving fails many times due to the name I give to the TIFF.

This is how I tried to do this:

        src=#original tiff with metadata
        numpy_res=#the array of the results after the analysis
        tiff_name='results/test7/2021-01-10/246/534514d2a593107sat31ecnvy4u0e1694k/results.tiff'
      
        
        with rasterio.open(tiff_name, 
                           'w',
                           driver='GTiff',
                           height=numpy_res.shape[0],
                           width=numpy_res.shape[1],
                           count=1,
                           dtype=numpy_res.dtype,
                           crs=src.crs,
                           nodata=None, # change if data has nodata value
                           transform=src.transform) as dst:
            dst.write(numpy_res, 1)

this is fails with next error :

RasterioIOError: Attempt to create new tiff file 'results/test7/2021-01-10/246/534514d2a593107sat31ecnvy4u0e1694k/results.tiff' failed: No such file or directory

now , I have to say that I do have folder called results, which is empty folder tha suppose to save the resulted array.
I have also try to change result to another name of folder that is not exists yet but that didn't work.

The weird thing is that if I give it simple name without the directory, such as:

'test.tiff'

it does work, but then it does not save it to the folder I want. Also, if I write the directory as shwn in the docomentrt, the:

/tmp/new.tif

it does not throw any error, though it also does not create new folder 'tmp' and the result cannot be found.

My end goal is to be able to save the results into the folder I specify.

Answer 1

There are two things going on here I think:

1. there is a difference between relative and absolute paths: /tmp/ is an absolute path, relative to the root directory. results/... is relative to the directory you are in. This is the same as ./results
2. indeed, you cannot store a file to a folder that doesn't exist yet. thread In stead, you should:

• create the directory
• add the file

3. please do not use hard-coded paths, this will give lots of headaches across platforms. In stead, Pathlib is a great library.

    import pathlib
    savedir = pathlib.Path('./results/test7/2021-01-10/246/534514d2a593107sat31ecnvy4u0e1694k/')
    #create the dir, including nested dirs and don't complain if it is already there
    savedir.mkdir(parents=True,exists_ok=True)
    #save the file
    with rasterio.open(savedir / 'results.tiff', 
                               'w',
                               driver='GTiff',
                               height=numpy_res.shape[0],
                               width=numpy_res.shape[1],
                               count=1,
                               dtype=numpy_res.dtype,
                               crs=src.crs,
                               nodata=None, # change if data has nodata value
                               transform=src.transform) as dst:
                dst.write(numpy_res, 1)