0

I am having issues using some modis hdf files in R. Working on Windows 10. I had a previous question where gdal wasn't working but I think I have resolved that. I tested with existing tifs and they work, however these modis files are not.

library(dismo)
library(MODISTools)
library (raster)
library(sp)
library(ggmap)
library(rgdal)
library(rgeos)
library(RCurl)
library(stringr)
library(gdalUtils)
require(gdalUtils)
hdf_dir <- list.files(workd, pattern="hdf$", full.names=FALSE) #create 

list of names with hdf extension
fhdf <-list.files(workd, pattern="hdf$", full.names=TRUE)
#create loop using gdal_translate
#test with one file
infile <- hdf_dir[1]
#"MOD13A3.A2010001.h11v04.005.2010043091608.hdf"
namefi <- substring(hdf_dir[1],1,nchar(hdf_dir[1])-4)
outfile <- paste0(out_tiff,"\\", namefi,".tif")
gdal_info <- GDALinfo(infile)

Error in .local(.Object, ...) : `C:\Users\R\MOD13A3.A2010001.h11v04.005.2010043091608.hdf' not recognised as a supported file format.

And:

gdal_translate(infile, namefi, sd_index=1)

Error in split1[[1]] : subscript out of bounds In addition: Warning message: running command '"C:\Program Files (x86)\GDAL\gdalinfo.exe" "MOD13A3.A2010001.h11v04.005.2010043091608.hdf"' had status 1

In the cmd line gdalinfo --formats gives:

 C:\Program Files (x86)\GDAL>gdalinfo --format hdf4
    Format Details:
      Short Name: HDF4
      Long Name: Hierarchical Data Format Release 4
      Extension: hdf
      Help Topic: frmt_hdf4.html
      Supports: Subdatasets

C:\Program Files (x86)\GDAL>gdalinfo --formats
Supported Formats:
  BAG (ro): Bathymetry Attributed Grid
  FITS (rw+): Flexible Image Transport System
  GMT (rw): GMT NetCDF Grid Format
  HDF4 (ros): Hierarchical Data Format Release 4
  HDF4Image (rw+): HDF4 Dataset
  HDF5 (ros): Hierarchical Data Format Release 5
  HDF5Image (ro): HDF5 Dataset
  netCDF (rw+s): Network Common Data Format
  VRT (rw+v): Virtual Raster
  GTiff (rw+vs): GeoTIFF
  NITF (rw+vs): National Imagery Transmission Format
  RPFTOC (rovs): Raster Product Format TOC format
  ECRGTOC (rovs): ECRG TOC format
  HFA (rw+v): Erdas Imagine Images (.img)
  SAR_CEOS (rov): CEOS SAR Image
  CEOS (rov): CEOS Image
  JAXAPALSAR (rov): JAXA PALSAR Product Reader (Level 1.1/1.5)
  GFF (rov): Ground-based SAR Applications Testbed File Format (.gff)
  ELAS (rw+v): ELAS
  AIG (rov): Arc/Info Binary Grid
  AAIGrid (rwv): Arc/Info ASCII Grid
  GRASSASCIIGrid (rov): GRASS ASCII Grid
  SDTS (rov): SDTS Raster
  DTED (rwv): DTED Elevation Raster
  PNG (rwv): Portable Network Graphics
  JPEG (rwv): JPEG JFIF
  MEM (rw+): In Memory Raster
  JDEM (rov): Japanese DEM (.mem)
  GIF (rwv): Graphics Interchange Format (.gif)
  BIGGIF (rov): Graphics Interchange Format (.gif)
  ESAT (rov): Envisat Image Format
  BSB (rov): Maptech BSB Nautical Charts
  XPM (rwv): X11 PixMap Format
  BMP (rw+v): MS Windows Device Independent Bitmap
  DIMAP (rov): SPOT DIMAP
  AirSAR (ro): AirSAR Polarimetric Image
  RS2 (ros): RadarSat 2 XML Product
  PCIDSK (rw+v): PCIDSK Database File
  PCRaster (rw): PCRaster Raster File
  ILWIS (rw+v): ILWIS Raster Map
  SGI (rw+): SGI Image File Format 1.0
  SRTMHGT (rwv): SRTMHGT File Format
  Leveller (rw+): Leveller heightfield
  Terragen (rw+): Terragen heightfield
  ISIS3 (rov): USGS Astrogeology ISIS cube (Version 3)
  ISIS2 (rw+v): USGS Astrogeology ISIS cube (Version 2)
  PDS (rov): NASA Planetary Data System
  TIL (rov): EarthWatch .TIL
  ERS (rw+v): ERMapper .ers Labelled
  JP2OpenJPEG (rwv): JPEG-2000 driver based on OpenJPEG library
  L1B (rovs): NOAA Polar Orbiter Level 1b Data Set
  FIT (rwv): FIT Image
  GRIB (rov): GRIdded Binary (.grb)
  RMF (rw+v): Raster Matrix Format
  WCS (rovs): OGC Web Coverage Service
  WMS (rwvs): OGC Web Map Service
  MSGN (ro): EUMETSAT Archive native (.nat)
  RST (rw+v): Idrisi Raster A.1
  INGR (rw+v): Intergraph Raster
  GSAG (rwv): Golden Software ASCII Grid (.grd)
  GSBG (rw+v): Golden Software Binary Grid (.grd)
  GS7BG (rw+v): Golden Software 7 Binary Grid (.grd)
  COSAR (ro): COSAR Annotated Binary Matrix (TerraSAR-X)
  TSX (rov): TerraSAR-X Product
  COASP (ro): DRDC COASP SAR Processor Raster
  R (rwv): R Object Data Store
  MAP (rov): OziExplorer .MAP
  PNM (rw+v): Portable Pixmap Format (netpbm)
  DOQ1 (rov): USGS DOQ (Old Style)
  DOQ2 (rov): USGS DOQ (New Style)
  ENVI (rw+v): ENVI .hdr Labelled
  EHdr (rw+v): ESRI .hdr Labelled
  GenBin (rov): Generic Binary (.hdr Labelled)
  PAux (rw+): PCI .aux Labelled
  MFF (rw+): Vexcel MFF Raster
  MFF2 (rw+): Vexcel MFF2 (HKV) Raster
  FujiBAS (ro): Fuji BAS Scanner Image
  GSC (rov): GSC Geogrid
  FAST (rov): EOSAT FAST Format
  BT (rw+v): VTP .bt (Binary Terrain) 1.3 Format
  LAN (rw+v): Erdas .LAN/.GIS
  CPG (ro): Convair PolGASP
  IDA (rw+): Image Data and Analysis
  NDF (rov): NLAPS Data Format
  EIR (rov): Erdas Imagine Raw
  DIPEx (rov): DIPEx
  LCP (rwv): FARSITE v.4 Landscape File (.lcp)
  GTX (rw+v): NOAA Vertical Datum .GTX
  LOSLAS (rov): NADCON .los/.las Datum Grid Shift
  NTv2 (rw+vs): NTv2 Datum Grid Shift
  CTable2 (rw+v): CTable2 Datum Grid Shift
  ACE2 (rov): ACE2
  SNODAS (rov): Snow Data Assimilation System
  KRO (rw+v): KOLOR Raw
  ARG (rwv): Azavea Raster Grid format
  RIK (ro): Swedish Grid RIK (.rik)
  USGSDEM (rwv): USGS Optional ASCII DEM (and CDED)
  GXF (ro): GeoSoft Grid Exchange Format
  HTTP (ro): HTTP Fetching Wrapper
  NWT_GRD (rov): Northwood Numeric Grid Format .grd/.tab
  NWT_GRC (rov): Northwood Classified Grid Format .grc/.tab
  ADRG (rw+vs): ARC Digitized Raster Graphics
  SRP (rovs): Standard Raster Product (ASRP/USRP)
  BLX (rw): Magellan topo (.blx)
  Rasterlite (rws): Rasterlite
  PostGISRaster (rws): PostGIS Raster driver
  SAGA (rw+v): SAGA GIS Binary Grid (.sdat)
  KMLSUPEROVERLAY (rwv): Kml Super Overlay
  XYZ (rwv): ASCII Gridded XYZ
  HF2 (rwv): HF2/HFZ heightfield raster
  PDF (rw): Geospatial PDF
  OZI (rov): OziExplorer Image File
  CTG (rov): USGS LULC Composite Theme Grid
  E00GRID (rov): Arc/Info Export E00 GRID
  ZMap (rwv): ZMap Plus Grid
  NGSGEOID (rov): NOAA NGS Geoid Height Grids
  MBTiles (rov): MBTiles
  IRIS (rov): IRIS data (.PPI, .CAPPi etc)

I can't figure out why this isn't working or what else I can try?

1

GDALinfo is a function in rgdal, and unless you compiled rgdal yourself against that installation of GDAL it won't have HDF4 - presumably you have the CRAN Windows binary, which isn't built with HDF (and some others). gdal_translate is not a function in rgdal, but you don't say what package/s you are using, so I don't know what that is (gdalUtils?). gdalUtils spawns to the system-installation of GDAL, as a workaround - "loose-coupling" rather than the "tight (but limited) coupling" in Windows rgdal.

This is all unfortunate but there it is, the new R-hub project may in future provide the capability to build rgdal with extra drivers on Windows. I just use rgdal on Linux rather than try to build on Windows any more.

EDIT: Focus on gdalUtils and how it works, without mixing in the rgdal as here - or just use the command-line directly. Windows file paths are also a pain, but you should be able to make it work. To help we need at least gdalinfo on the file, and perhaps on the subdataset you are after. Pretty sure gdalUtils will just create geotiffs, so try 'gdal_translate input.hdf output.tif -sds' to spit out all of them to a format raster() can read (forget about rgdal, let raster drive that). You still have manual georeferencing, possibly with geolocation arrays and so on to workaround with this format.

This is a very specialist file, so you probably need help from domain experts.

  • added more info to post - any ideas on what to do? I'd rather not create a VM for linux at this time tbh.... – GISKid Jun 23 '16 at 14:45
  • I think that's the far easier pathway here. Reading this with raster() will shortcut you immensely. – mdsumner Jun 23 '16 at 14:50
0

After much searching, I think I no longer have any errors when running these tools. First, I deleted a few gdal installs on my computer and kept the 32-bit version. Then I followed the steps outlined here and set my environment paths to where my GDAL install was located. This fixed the error in rgdal but not when using gdalUtils. To get gdalUtils to work I had to set the gdal installation in R where hdf was supported. Here is an example of my working code:

HDFd <- "C:\\Users\\L\\R\\hdfs"
out_tiff <- "C:\\Users\\L\\R\\Tiffs"
hdf_dir <- list.files(HDFd, pattern="hdf$", full.names=FALSE) #create list of names with hdf extension
#Create full directory name of files
fhdf <-list.files(HDFd, pattern="hdf$", full.names=TRUE)
#create loop using gdal_translate
#test with one file
#use full path
infile <- fhdf_dir[1]
#"MOD13A3.A2010001.h11v04.005.2010043091608.hdf"
gdal_chooseInstallation(hasDrivers=c("HDF4","HDF5"))
# Get the version of this installation:
getOption("gdalUtils_gdalPath")[[
  gdal_chooseInstallation(hasDrivers=c("HDF4","HDF5"))]]$version

namefi <- substring(hdf_dir[1],1,nchar(hdf_dir[1])-4)
outfile <- paste0(out_tiff,"\\", namefi,".tif")
sds <- get_subdatasets(fhdf[1])
#Use full directory name
gdal_translate(fhdf[1], outfile, sd_index=1)
#For loop to convert all to TIFF in directory
for (i in 1:length(fhdf)){
  name <-substring(hdf_dir[i],1,nchar(hdf_dir[i])-4)
  newfile <- paste0(out_tiff,"\\", name,".tif")
  print(name)
  gdal_translate(fhdf[i],newfile,sd_index=1)

}

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.