How to download and process raster files in Python?

Question

I am fairly new to python and seek guidance for a question which might sound trivial to many. Is there a way to use 'wget' in a python script to download raster files from a server and process them in the same script?

Answer 1

Python has urllib2 built-in, which opens a file-pointer-like object from a IP resource (HTTP, HTTPS, FTP).

import urllib2, os

# See http://data.vancouver.ca/datacatalogue/2009facetsGridSID.htm
rast_url = 'ftp://webftp.vancouver.ca/opendata/2009sid/J01.zip'
infp = urllib2.urlopen(rast_url)

You can then transfer and write the bytes locally (i.e., download it):

# Open a new file for writing, same filename as source
rast_fname = os.path.basename(rast_url)
outfp = open(rast_fname, 'wb')

# Transfer data .. this can take a while ...
outfp.write(infp.read())
outfp.close()

print('Your file is at ' + os.path.join(os.getcwd(), rast_fname))

Now you can do whatever you want with the file.

Answer 2

A couple of ways to accomplish this. You can use the subprocess module to call wget - see http://docs.python.org/library/subprocess.html

import subprocess

retcode = subprocess.call(["wget", args])

Or you can use python to download the file directly using the urllib (or urllib2) module - http://docs.python.org/library/urllib.html. There are examples in the documentation.

Answer 3

In this previous answer is a method using a call to os.system.

os.system('wget %s' % (fullurl))