Web mapping software for a huge raster time series?

Question

I'm an ArcGIS Desktop user who is stepping into the world of web mapping for the first time. Following advice I found here on GIS.SE, I started with the free tutorials from OpenGeo.

However, I began to realize that most of the demos and examples were geared towards vector data. My main project involves the display of a 300 frame time series of 5000 x 5000 pixel rasters. On my desktop computer these are stored in a single 5000x5000x300 16-bit integer BIP binary, which totals around 8GB. My goal is to be able to click on a cell (of a single raster in the time series) and have a graph pop up showing the values of that pixel in the 300-element time series. The raster data used for each time series graph must be stored losslessly, though the overlay maps that are clicked on can be lossy caches.

Is there anything that may be better suited for this project (for a novice web GIS developer) than OpenGeo? Or should I just keep going with this setup?

For reference, I have programming experience in Python, Java, and PHP. I don't have much experience with SQL. This is an open-ended project so I have plenty of time to learn new languages if need be. I already have a web server, though I'll probably have to switch hosts because GoDaddy doesn't seem to support PostgreSQL without getting a VPS.

Thanks for your help!

EDIT: (Jan 13) I'm still seeking information on exactly how best to store a 3 dimensional 16-bit integer BIP raster and be able to efficiently query a single z-axis "column" of data. I don't want to convert it to a 32-bit format (because that would double its file size from its current 16-bit form).

Answer 1

EDIT: (Jan 13) I'm still seeking information on exactly how best to store a 3 dimensional 16-bit integer BIP raster and be able to efficiently query a single z-axis "column" of data. I don't want to convert it to a 32-bit format (because that would double its file size from its current 16-bit form).

Querying such a raster should not really pose large problems. You can read binary data directly using all program languages, and access is fast. Just make sure that you store your data in a file format which has all metadata in a seperate file. BIP is such a format

eg in php, assuming that the file is row major order (otherwise switch x and y), with $x and $y the position in your grid(counting from 0), $nx, $ny and $nz the number of pixels in each dimension and $nb the number of bytes per gridcell:

$fp = fopen('yourfile.bil', 'r');

fseek ($fp, $nz*$nb*($y*$nx +$x))//this is a very fast operation
// read some data
$data = fread($fp, $nz*nb);//this is also very fast

Just make sure that you access the right pixel: does counting start from top left or not, ...

Some extra info: After reading the data, you should convert it to floats. E.g.:

$dataf=unpack("f*", $data);
print_r($dataf);

In case your host does not support uploading large files, you could eg split up your bip file in eg 8 bip files.

Some more info on how I would do the rest of the website: since your data is static, you could generate a small mapviewer using gdal2tiles and openlayers. http://www.gdal.org/gdal2tiles.html In fact, since you say that "I don't really need scalability--this project is mostly to enable a few peer researchers to have easier access to my data than sending an 8GB file and loading it in ENVI."you could maybe even do without using a webgis toolbox: just let your users click on the image and catch the coordinates: http://www.emanueleferonato.com/2006/09/02/click-image-and-get-coordinates-with-javascript/

(though you should find a way to present your 5000x5000 image nicely)

Answer 2

This looks like three separate issues: one of infrastructure, one of architecture and one of event handling. I'll lay out one possible approach, but my answer is necessarily general.

Infrastructure

I recommend using a VPS hosting service like Linode (www.linode.com) for your server. This give you full (i.e., root) access to a professionally maintained server - no worries of the power going out or losing its connection to the Internet.

Architecture

There are so many options here that it can be really overwhelming. As an example, I run a couple of systems with GeoServer and OpenLayers. GeoServer is served up by Tomcat 7. The OpenLayers/jQuery front end is served up by Apache2. You can include Postgres/PostGIS for storing vector data, but this is not a good option for raster data. You could also set up a Python system using Django or even web.py (http://webpy.org/) for a rather simple controller. GeoServer lets you store raster data in the following formats:

• ArcGrid - Arc Grid Coverage Format
• GeoTIFF - Tagged Image File Format with Geographic information
• Gtopo30 - Gtopo30 Coverage Format
• ImageMosaic - Image mosaicking plugin
• WorldImage - A raster file accompanied by a spatial data file

Event Handling

When the user clicks on the map you want to pop up a time-series plot of the field data at that point. First, set up a controller, which could be written in Python or Java, that listens for url requests with a latitude and longitude. This controller returns either a static image rendered on the server or json data that the client (jQuery) can turn into a plot.

Next, to get the XY data on the map you might use a function like this (see http://dev.openlayers.org/releases/OpenLayers-2.11/examples/click-handler.html):

onClick: function(evt) {
    var clickSpot = evt.xy;
    // pass clickSpot to the controller ...
}

I hope that helps.

Answer 3

If I understand correctly your question, I would create a time WMS service, for example with MapServer.

This way it will be easy to display the correct raster for each datetime (with a GetMap request), and request the values for a cell in a given datetime range (with a GetFeatureInfo request).