Programmatically retrieving Landsat 8 Satellite Images?

Question

I have not been able to find out this myself, but is there a way to programmatically access Landsat 8 Satellite Images?

I know that I can go to USGS EarthExplorer to search and browse through the scenes, and even download them. But I do not want to download them to my own computer for two reasons, My internet is slow and my laptop have cooling issues.

So instead I figure I want to transfer the images directly to a “cloud”-instance for data processing. I would also be nice if I could download and process these images as they become available from USGS.

Answer 1

I saw a blog post from developmentseed for their command line utility landsat-util.

Power tools for Satellite Imagery

The landsat-util can be forked from github and compiled from source unless your OS offers it in a binary ready to go.

The blog describes it simply as:

a command line utility that makes it easy to search, download, and process Landsat imagery.

You can search based on date, cloud coverage % and other things, download immediately, or process once it's downloaded like pansharpen or stitch the images together.

You can preview images before you download. Search commands provide a link to a thumbnail for each image.

landsat search --cloud 4 --start "August 1 2013" --end "August 25 2014" country 'Vatican'

Using the --pansharpen flag will take longer to process but will produce clearer images.

landsat search --download --imageprocess --pansharpen --cloud 4 --start "august 11 2013" --end "august 13 2013" pr 191 031

You can also perform all processing on images that you previously downloaded.

landsat download LC81050682014217LGN00

landsat process --pansharpen /your/path/LC81050682014217LGN00.tar.bz

Here's a readme with more info.

Answer 2

How about firing up an EC2 or rackspace instance and installing the EarthExplorer bulk download application:

http://earthexplorer.usgs.gov/bulk/

You could hit the EarthExplorer service with a POST request to submit jobs programmatically:

http://earthexplorer.usgs.gov/subscription/submit/

You would need to provide standingRequestName, frequency, subscription_start, subscription_end, search_start, search_end parameters. This would get you started, but it feels like a back door and that the bulk downloader application running on an EC2 instance would be a better and less time consuming option.

Answer 3

I intend to do the same so I start an Amazon EC-2 instance and install the Bulk Download on it. But as far as I know it's a graphical application and nothing in the doc Bulk Download Tutorial lets hope that you can use it with the terminal.

I read here about the possibility of using Curl but it returns an 403 access denied

---

After writing emails to USGS, the official answer is that Bulk Download is the only way for massive download.

But Charlie Loyd From Mapbox wrote me another possibility:

Google Storage also mirrors much of the Landsat archive. You can install their “gsutil” (free) and then list this directory, which is indexed by sensor (for example, L8 is Landsat 8) and path/row: gsutil ls gs://earthengine-public/landsat/

I have to add that when browsing through paths and rows remember that it is a 3 numbers based. For example if you need Path 210 Row 40, you have to browse for folder 210 and then 040.

Answer 4

I wrote a bash script to do this. Note you can customize the url and mouse locations and button clicks in the segment of code with the xdotool command being utilized. Here is the source code:

via: Everything Awesome YTC in the bash playlist

#!/bin/bash

echo " "

sudo apt-get install xdotool -y
sleep 2

echo " "
echo "# NASA/NOAA/NHC Hurricane Satellite Imagery Bot [H.S.I.B]"
echo "# Coded by Scott Hermann "
echo "[ YouTube: https://www.youtube.com/user/postulatedstate/videos/ ]"
echo " "

# * * Instructions * *
# 1] Click on map to select area to monitor.
# 2] Set delay variable ( time between saves )
# The script will now refresh and save the images-
# -every X seconds, and save using the filenames
# filename-X.jpg; filename-X.jpg; etc..

export PS1="\e[0;30m[\u@\h \W]\$ \e[m "
# make black prompt - color code=[0;30]

chromsat_loop()
{
 chromium -new-tab https://weather.msfc.nasa.gov
 sleep 2

 notify-send "Your browser will now open the NASA website. Select the section of earth to monitor via Satallite by clicking on it, and then enter Y or N (caps) into the terminal, or press the [Begin] button on the GUI."
 echo " "
}


get_settings()
{
  echo -n "Enter storm name: "; read stormname
  echo -n "Season? [ Ex: 2017 ] "; read season
  echo -n "Month "; read month
  # echo -n "data"; <- remember ;
  sleep 2
  echo " "
  echo "Checking to see if a database for hurricane $stormname of the $season season exist in the month of $month"
  sleep 2
  echo " "
  # if dir $season = true then confirm
  if [ -d /root/Desktop/$season ]; then
     notify-send "The hurricane season exist in the database."
     echo "The hurricane season exist in the database."
     echo " "
  else
     echo "It doesn't appear that this season is in the database. I will create it for you."
     mkdir /root/Desktop/$season
  fi

  # if dir $season/$month = true then confirm

  if [ -d /root/Desktop/$season/$month ]; then
     notify-send "The month entered has data in the database."
     echo "The month entered has data in the database."
     echo " "
  else
     mkdir /root/Desktop/$season/$month
  fi

  # if dir $season/$month/$stormname = true then confirm

  if [ -d /root/Desktop/$season/$month/$stormname ]; then
     notify-send "This storm is in the database."
     echo "This storm is in the database."
  else
     echo "Adding the storm to the database."
     notify-send "Adding the storm to the database."
     echo " "
     sleep 1
     mkdir /root/Desktop/$season/$month/$stormname
  fi
  echo " "
  sleep 2

  echo -n "Enter data retrieval delay variable: ( 1200 = 20 minutes ) "; read delay

  echo " "
  sleep 2

  echo " "
  echo "1] Chromium"
  echo "2] Firefox"
  echo "3] Qupzilla"
  echo "4] Modori"
  echo " "
  echo -n "Which Web-Browser are you using? "; read browser


  if [ "$browser" = "1" ]; then

       echo "Initializing NASA Data Transfer..."
       chromsat_loop
  fi

  if [ "$browser" = "2" ]; then

       echo "$browser set to Firefox"
  fi


# End Of Function
}

get_settings

# Note: Calling functions in bash -> fun_ction

# Create 4 functions for the 4 browsers it will support..



ready_ornot()
{
  notify-send "In 45 seconds, the script will begin data retrieval.. Go ahead and select the portion of the storm on the map that you wish to monitor. Save the first image to the directory on your desktop. It's listed under the year, the month, and the storm. From there, the bot will save the images to that folder."
  echo "In 45 seconds, the script will begin data retrieval.. Go ahead and select the portion of the storm on the map that you wish to monitor. It's listed under the year, the month, and the storm. From there, the bot will save the images to that folder."

  newloop=0
  until [ "$newloop" = "45" ]
  do
     sleep 1
     echo "Initializing data retrieval process in 45 seconds. @ $newloop second(s)...."
     newloop=`expr $newloop + 1`
  done

  notify-send "Initializing data retrieval process.. "
  echo "Initializing data retrieval process.."
  echo " "

}

ready_ornot

begin_dataretr()
{
 sleep 2
 notify-send "@ Data Retrieval Function."
 # url opened in new tab waiting
 # variables=value; = NO space [x]-> var = 1<-(wrong)
 forever=1
 looper=1
 while [ "$forever" == "1" ];
 do
     # refresh chromium page
     sleep $delay && echo $delay # 60 for test; 1800=30 mins
     # Save images after refresh;
     # plug\/ looper \/ variable in filenames
     # file1.jpg file2.jpg file3.jpg etc.


     # <!--#!/bin/bash-> to be modified and completed * * * * * * * * * #!/bin/bash -!>


     # CD to folder 
     # /root/Desktop/season/month/stornname
     # ( that should make the save as file dialog box open in our folder )
     echo " " 
     echo "Done sleeping.."
     echo " "
     # xdotool getmouselocation --shell

     xdotool mousemove 765 438
     # mouse positioned to click on save file box
     # for loop through var to save file name as
     sleep 1
     xdotool click 1
     # text box clicked and waiting.
     sleep 1
     xdotool key F5
     sleep 7
     xdotool mousemove 953 229
     sleep 1
     xdotool click 1
     sleep 5
     xdotool click 3
     sleep 3
     xdotool key Down
     xdotool key Down
     sleep 2
     xdotool mousemove 983 261
     sleep 1
     xdotool click 1
     sleep 1
     # xdotool key Enter
     sleep 1   
     xdotool mousemove 1146 763
     sleep 1
     xdotool click 1
     notify-send "New data retrieval has been saved to database.."

     # <!--#!/bin/bash-> to be modified and completed * * * * * * * * * #!/bin/bash -!>

 done
}
begin_dataretr
# no () when calling functions.
#  - - - - - -  - only when def them.

Answer 5

https://github.com/olivierhagolle/LANDSAT-Download

This script, with some tweaking (indentation issues and some comments/variables were in french) has been extremely helpful to me. However, it can only download the data that's already in the archive. So for data that hasn't been processed, you have to order it. I'm working on automating that now, but off to a slow start (as i just started 30 minutes ago). Eventually I want the data to be downloaded to my cloud server, but for now I'm just testing it on my desktop (you just specify an output directory and the files are automatically put there).

I too am interested in downloading the data as it becomes available--specifically the surface reflectance data before it is thrown out (I'm told USGS processes the raw data into surface reflectance temporarily before throwing it out, although some gets stored in the archive). Please let us know if you have made any headway.

Answer 6

Esta plataforma é simples ! faça download do KMl e abra-o no google earth . Pesquise um lugar e achara o path/row (lugar central de possiveis imagens) ponha o path/row que achou para o lugar e ponha na plataforma.

Todas as imagens para aquele local aparecera

Google Translation:

This platform is simple! download KML and open it in google earth. Search for a place and find the path / row (central place of possible images) put the path / row you found for the place and put it on the platform. All images for that location will appear

http://georef.vantis.pt