I'm trying to convert a .tif to .mrf format using Python gdal bindings. I'm doing the conversion in memory using /vsimem then attempting to write directly to s3. The problem is only the .mrf file gets copied to s3 but none of the auxiliary side car files.

When I write to disk it writes a few side car files like .idx, lrc, .mrf.aux.xml which are necessary. It does this without me having to explicitly tell it to output those files.

How would I go about accomplishing this in s3? How do I know what files need to be copied to s3? This works when I just have a single output file.

from osgeo import gdal
import boto3

s3 = boto3.resource('s3')
object = s3.Object('test-bucket', 'testing/output.mrf')

tif_file = r"C:\new_folder\test.tif"
ds = gdal.Open(tif_file)

gdal.Translate('/vsimem/temp_file.mrf', ds, format="MRF", creationOptions=['COMPRESS=NONE', 'BLOCKSIZE=10'])

f = gdal.VSIFOpenL('/vsimem/temp_file.mrf', 'rb')
gdal.VSIFSeekL(f, 0, 2)  # seek to end
size = gdal.VSIFTellL(f)
gdal.VSIFSeekL(f, 0, 0)  # seek to beginning
data = gdal.VSIFReadL(1, size, f)


ds = None

Got some help and figured it out. Changed some things around and basically need to

use gdal.ReadDir('/vsimem/temp/') to get a list of all files created and then add those files one by one to s3.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.