I've got what seems to be a common problem regarding HDF files, but after spending a couple days going around in circles with R, GDAL, etc., I'm turning here for (hopefully) some insight.
My goal is to make a GeoTiff raster which is a mosaic of a bunch of different HDF input files. I started by trying to generally follow the workflow described on this tutorial, which is:
- Convert HDF files to GeoTiffs and reproject them.
- Mosaic reprojected GeoTiffs.
However, I am having significant difficulties with Step 1, specifically that I can't figure out how to open/work with the HDF files. The raw input datafiles are estimates of Leaf Area Index from the GLASS LAI dataset, derived from MODIS satellite data, and are described here and available for download here.
Following the tutorial linked above, I have installed the
gdalUtils package for R, and attempted to get some information about the files, but this returned a somewhat lengthy error:
require(gdalUtils) gdalinfo("C:/.../001/GLASS01A01.V03.A2010001.h17v03.2012253.hdf")  "ERROR 4: `C:/.../001/GLASS01A01.V03.A2010001.h17v03.2012253.hdf' not recognized as a supported file format."  ""  "gdalinfo failed - unable to open 'C:/.../001/GLASS01A01.V03.A2010001.h17v03.2012253.hdf'." attr(,"status")  1 Warning message: running command '"C:\Program Files\QGIS 2.16\bin\gdalinfo.exe" "C:/.../001/GLASS01A01.V03.A2010001.h17v03.2012253.hdf"' had status 1
After some research, I began to suspect that the
not recognized as a supported file format error was occurring because the file is an HDF4 file, not an HDF5 file. Based on this previous question, I attempted to select an installation that works with HDF4 files, but this also returned an error:
gdal_chooseInstallation(hasDrivers=c("HDF4","HDF5")) Error in gdal_chooseInstallation(hasDrivers = c("HDF4", "HDF5")) : No installations match. In addition: Warning message: In grep(paste(hasDrivers, "-", sep = ""), installation_drivers$format_code) : argument 'pattern' has length > 1 and only the first element will be used
To remedy this, I attempted to install a new version of GDAL (gdal-201-1800-x64-ecw-33.msi) following the tutorial here. However, the same errors as above occurred when using R.
When I attempt to do things directly from the Windows command line, I can verify that gdal is installed and knows about the hdf4 format:
> C:\Users\Sam>gdalinfo --format hdf4 Format Details: Short Name: HDF4 Long Name: Hierarchical Data Format Release 4 Supports: Raster Extension: hdf Help Topic: frmt_hdf4.html Supports: Subdatasets Supports: Open() - Open existing dataset.
However, when I try to run
gdalinfo via the command line, I get a comparable error:
C:\Users\Sam>gdalinfo C:\...\001\GLASS01A01.V03.A2010001.h17v03.2012253.hdf ERROR 4: `C:\...\001\GLASS01A01.V03.A2010001.h17v03.2012253.hdf' not recognized as a supported file format. gdalinfo failed - unable to open 'C:\...\001\GLASS01A01.V03.A2010001.h17v03.2012253.hdf'.
At this point, I am not sure whether my file is actually HDF4 and I'm just doing something wrong, or there is an issue with the file itself, or a problem with my GDAL installation, or some other option I'm not aware of.
I am running 64-bit Windows 10 and have the 2013 (12.0.30501) version of C++ installed.
The problem seems to be associated with the manner in which I downloaded the files. I used a loop in R to acquire several hundred of these images with the
download.file command. However, if I manually go to the FTP website, right click on a file name, and do
Save as... to save it to my computer,
gdalinfo works correctly.
Note that the files are also slightly different sizes. The one downloaded using R is 374 kb, and the one downloaded by hand is 373 kb. The permissions of the two files appears to be the same. Is there a way to use
download.file that replicates the
Save as... functionality? I've also tried re-downloading all the .hdf files in R, with and without their .xml accompaniments, with the same issue. I need to download several hundred files and would prefer not to do it by hand!
Selecting a different
method flag in
download.file fixed the problem. Previously I was using the default method, which for Windows turns out to be
wininet. When I set the
method="curl" flag in my call to
download.file, GDAL worked correctly on the newly downloaded files. I have added an answer below explaining this.