2

I am currently working with GOES-R on AWS. I've downloaded recent data, which is in the netCDF file format. Specifically I am using band 3 (M3C03).

When loading it into GDAL, it is taking upwards of 5 minutes.

goes_dir = "goes"
goes_files = glob.glob(goes_dir + '/' + str(doy) + '/18/*M3C03*.nc')
goes_files[:3]

['goes/257/18/OR_ABI-L1b-RadF-M3C03_G16_s20172571815379_e20172571826146_c20172571826188.nc',
 'goes/257/18/OR_ABI-L1b-RadF-M3C03_G16_s20172571845379_e20172571856146_c20172571856196.nc',
 'goes/257/18/OR_ABI-L1b-RadF-M3C03_G16_s20172571830379_e20172571841146_c20172571841193.nc']

go = gdal.Open('NETCDF:' + goes_files[1] + ':Rad')
import time
start_time = time.time()
goes_arr = go.GetRasterBand(1).ReadAsArray()
print("--- %s seconds ---" % (time.time() - start_time))
# --- 277.96904397 seconds ---

What can I do to speed it up? The files are only 50 MB maximum.

Improve this question
6
  • 1
    You could look at using the netCDF library as shown here gis.stackexchange.com/questions/185352/… and also demonstrated to improve performance here lists.osgeo.org/pipermail/gdal-dev/2016-November/045573.html. That last link also suggests using GDAL translate to modify the raster so you could try that also
    – Liam G
    Commented Nov 7, 2017 at 0:33
  • 1
    Can you point some test data that can be downloaded directly?
    – user30184
    Commented Nov 7, 2017 at 7:31
  • I linked to the AWS page. That's as direct as it gets with AWS.
    – Matthew Mage
    Commented Nov 7, 2017 at 18:39
  • @Freighty netCDF4-python opens the file effortlessly. GDALWarp still takes 5+ minutes to warp the netCDF, which is understandable considering its still using GDAL under the hood.
    – Matthew Mage
    Commented Nov 7, 2017 at 18:41
  • Can you just use the netCDF4 library to get the data into a numpy array and then convert to tif or other format? Such as here gis.stackexchange.com/questions/37238/…
    – Liam G
    Commented Nov 8, 2017 at 2:44

1 Answer 1

Reset to default
1

It doesn't look like there's an easy way to fix this. My solution was to warp it to a known projection GeoTIFF, which takes several minutes but only needs to be done once.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.