2

I am currently working with GOES-R on AWS. I've downloaded recent data, which is in the netCDF file format. Specifically I am using band 3 (M3C03).

When loading it into GDAL, it is taking upwards of 5 minutes.

goes_dir = "goes"
goes_files = glob.glob(goes_dir + '/' + str(doy) + '/18/*M3C03*.nc')
goes_files[:3]

['goes/257/18/OR_ABI-L1b-RadF-M3C03_G16_s20172571815379_e20172571826146_c20172571826188.nc',
 'goes/257/18/OR_ABI-L1b-RadF-M3C03_G16_s20172571845379_e20172571856146_c20172571856196.nc',
 'goes/257/18/OR_ABI-L1b-RadF-M3C03_G16_s20172571830379_e20172571841146_c20172571841193.nc']

go = gdal.Open('NETCDF:' + goes_files[1] + ':Rad')
import time
start_time = time.time()
goes_arr = go.GetRasterBand(1).ReadAsArray()
print("--- %s seconds ---" % (time.time() - start_time))
# --- 277.96904397 seconds ---

What can I do to speed it up? The files are only 50 MB maximum.

  • 1
    You could look at using the netCDF library as shown here gis.stackexchange.com/questions/185352/… and also demonstrated to improve performance here lists.osgeo.org/pipermail/gdal-dev/2016-November/045573.html. That last link also suggests using GDAL translate to modify the raster so you could try that also – Liam G Nov 7 '17 at 0:33
  • 1
    Can you point some test data that can be downloaded directly? – user30184 Nov 7 '17 at 7:31
  • I linked to the AWS page. That's as direct as it gets with AWS. – Matthew Mage Nov 7 '17 at 18:39
  • @Freighty netCDF4-python opens the file effortlessly. GDALWarp still takes 5+ minutes to warp the netCDF, which is understandable considering its still using GDAL under the hood. – Matthew Mage Nov 7 '17 at 18:41
  • Can you just use the netCDF4 library to get the data into a numpy array and then convert to tif or other format? Such as here gis.stackexchange.com/questions/37238/… – Liam G Nov 8 '17 at 2:44
1

It doesn't look like there's an easy way to fix this. My solution was to warp it to a known projection GeoTIFF, which takes several minutes but only needs to be done once.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.