3

I am having problems with ModisDownload recently that I have not had when using it so far. I dowloaded, reprojected and mosaiced modis images about two months ago successfully, despite some errors during downloading, which I assumed were due to the NASA servers, reported sluggish at times.

However, for the last week it is not working in a new way... I get no errors during downloading, but the .hdf files downloaded all are size 520 bytes, which is clearly wrong. MRT is then unable to open the .hdf files downloaded to reproject, leading to an error.

I have not changed anything to my configuration or my internet connection (from home, so no institution limitations on ftp access), and tried updating packages already. Has anybody been experiencing the same error recently? Has something changed?

The code I have been using is below and is straightforward. I decomposed the historic download in loops because ModisDownload downloads first before reprojecting and mosaicing. I wanted avoid restarting from scratch every time there would be a downloading error. I tried with a product name rather than the full http address, but I get the same error.

setwd(paste0("/media/olivier/olivier_ext/gedata_current/jde_coffee/MODIS/",state.nm,"/raw_data"))
FTP <- "http://e4ftl01.cr.usgs.gov/MOLT/MOD13Q1.005/"
years <- c(2015:2006)
start <- c("01.01","07.01")
end <- c("06.30","12.31")
for (i in 1:length(years)) {
  for (j in 1:length(start)) {
    ModisDownload(x=FTP,h=c(13,14),v=c(10,11),dates=c(paste0(years[i],".",start[j]),paste0(years[i],".",end[j])),
              mosaic=T,MRTpath='/home/olivier/MRT/bin', #'C:/modis_MRT/bin' 
              UL=c(state_ex@xmin,state_ex@ymax),LR=c(state_ex@xmax,state_ex@ymin), 
              bands_subset="1 1 1 1 1 1 1 0 0 0 1 0", 
              proj=T,proj_type="GEO",proj_params="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0",datum="WGS84",pixel_size=0.002259948,
              delete=T)
  }
}

I am right now switching to downloading by hand and then using reprojectHDF and mosaicHDF, but it is quite inconvenient for regular upadtes with new MODIS images.

  • I'm having the same problem in downloading the same tiles as you. Have you figured out what was going wrong? Cheers – user80875 Aug 16 '16 at 13:39
  • Can you share the workaround that you did? Thank you! – vamalabanan Oct 27 '16 at 15:01
2

The problem was related to the new policy by NASA that an authentication is needed for any download from their data portals. The ModisDownload function in the rts package is now updated, and supports the new changes. Follow the help page of the function in the new version of the rts package, to see how you can set your username and password on your machine using setNASAauth function (only first time), and then use the ModisDownload function to download the images as before.

  • Works perfect. Thanks you for building an amazing package! – aldo_tapia Jan 3 '17 at 17:26
2

This has been an issue with the MODIS Reporjection Tool, as the files you are trying to download are placed in different folders on their server and the code often fails to parse through each of them.

Use the MODIS Data Download Manager through the LP DAAC web portal.

1) This link will take you to the portal with the download manager. Click on LPDAAC2Disk Download Manager. Here, you have two options - One, to download the manager or use the web interface.

2) I suggest clicking on the web interface. That will take you to this page.

3) Now draw a polygon around your region. Click on the dataset you need, the time period required and click submit. The download usually takes around 2-3 hours depending on the number of files and the size of the region selected.

4) One can open these files in ArcMap and use the Extract SubDataset tool to get either EVI or NDVI or the data that you need. Alternately, store all these files in a folder and use the python script below to perform the same.

 import arcpy,os,re


 arcpy.env.workspace = "F:\\MODIS Data\\"
 out_dir = "F:\\MODIS Data\\EVIData\\"


 rasters = arcpy.ListRasters("*")
 count = 1
 for raster in rasters:
    output = str(raster)
    s1 = raster.split('.')[2]
    s1 =s1[1:]
    name = out_dir + raster.split('.')[1] + s1
    print name


  arcpy.ExtractSubDataset_management(raster, name, "1")

5) Note, you are basically manually doing what the reprojection tool does. So you are extracting a subdataset and then you need to project it to the projection you need.

  • @Olivier Please click on the tick mark if the question has been answered. – Vijay Ramesh Aug 17 '16 at 13:47
  • Thanks for taking the time, but the whole point is to avoid doing it manually on the website. I implemented a workaround using RCurl to download the files, then using the separate functions to reproject and mosaic. – Olivier Oct 25 '16 at 10:19
2

should be aware that ftp is used while the page is http, this line should be reviewed carefully

FTP <- http://e4ftl01.cr.usgs.gov/MOLT/MOD13Q1.005/

0

Late to the party, but you can also use the get_modis tool. It's quite simple to use and deals with authentication and so on. You can use it from the command line as

./get_modis.py --username=<username> --password=<password> \
    -v -p MCD45A1.005 -s MOTA -y 2004 -t h17v04 -o /tmp/ 

This downloads the MCD45A1 (C5) burned area product for 2004 for tile h17v04 on to folder /tmp/. Obviously, you need to add your username and password for this to work.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.