Take the 2-minute tour ×
Geographic Information Systems Stack Exchange is a question and answer site for cartographers, geographers and GIS professionals. It's 100% free, no registration required.

I'm an ArcGIS Desktop user who is stepping into the world of web mapping for the first time. Following advice I found here on GIS.SE, I started with the free tutorials from OpenGeo.

However, I began to realize that most of the demos and examples were geared towards vector data. My main project involves the display of a 300 frame time series of 5000 x 5000 pixel rasters. On my desktop computer these are stored in a single 5000x5000x300 16-bit integer BIP binary, which totals around 8GB. My goal is to be able to click on a cell (of a single raster in the time series) and have a graph pop up showing the values of that pixel in the 300-element time series. The raster data used for each time series graph must be stored losslessly, though the overlay maps that are clicked on can be lossy caches.

Is there anything that may be better suited for this project (for a novice web GIS developer) than OpenGeo? Or should I just keep going with this setup?

For reference, I have programming experience in Python, Java, and PHP. I don't have much experience with SQL. This is an open-ended project so I have plenty of time to learn new languages if need be. I already have a web server, though I'll probably have to switch hosts because GoDaddy doesn't seem to support PostgreSQL without getting a VPS.

Thanks for your help!

EDIT: (Jan 13) I'm still seeking information on exactly how best to store a 3 dimensional 16-bit integer BIP raster and be able to efficiently query a single z-axis "column" of data. I don't want to convert it to a 32-bit format (because that would double its file size from its current 16-bit form).

share|improve this question
2  
Can you put PostgreSQL/PostGIS on your Webserver? - Would strongly recommend having your own server for unless your looking for scalable options then Amazon AWS (EC2 Relational Databases AMI) aws.amazon.com/running_databases might be an option. –  Mapperz Jan 12 '12 at 17:13
1  
My host (GoDaddy) doesn't support PostgreSQL according to this thread. I don't really need scalability--this project is mostly to enable a few peer researchers to have easier access to my data than sending an 8GB file and loading it in ENVI. –  dmahr Jan 12 '12 at 17:23
1  
If you are considering a new host, I cannot recommend webfaction highly enough; they offer postgresql/postgis1.5 but for raster functionality you probably want postgis2.0. This is on shared hosting too. –  djq Jan 13 '12 at 15:38
add comment

3 Answers 3

up vote 6 down vote accepted

EDIT: (Jan 13) I'm still seeking information on exactly how best to store a 3 dimensional 16-bit integer BIP raster and be able to efficiently query a single z-axis "column" of data. I don't want to convert it to a 32-bit format (because that would double its file size from its current 16-bit form).

Querying such a raster should not really pose large problems. You can read binary data directly using all program languages, and access is fast. Just make sure that you store your data in a file format which has all metadata in a seperate file. BIP is such a format

eg in php, assuming that the file is row major order (otherwise switch x and y), with $x and $y the position in your grid(counting from 0), $nx, $ny and $nz the number of pixels in each dimension and $nb the number of bytes per gridcell:

$fp = fopen('yourfile.bil', 'r');

fseek ($fp, $nz*$nb*($y*$nx +$x))//this is a very fast operation
// read some data
$data = fread($fp, $nz*nb);//this is also very fast

Just make sure that you access the right pixel: does counting start from top left or not, ...

Some extra info: After reading the data, you should convert it to floats. E.g.:

$dataf=unpack("f*", $data);
print_r($dataf);

In case your host does not support uploading large files, you could eg split up your bip file in eg 8 bip files.

Some more info on how I would do the rest of the website: since your data is static, you could generate a small mapviewer using gdal2tiles and openlayers. http://www.gdal.org/gdal2tiles.html In fact, since you say that "I don't really need scalability--this project is mostly to enable a few peer researchers to have easier access to my data than sending an 8GB file and loading it in ENVI."you could maybe even do without using a webgis toolbox: just let your users click on the image and catch the coordinates: http://www.emanueleferonato.com/2006/09/02/click-image-and-get-coordinates-with-javascript/

(though you should find a way to present your 5000x5000 image nicely)

share|improve this answer
    
Awesome, this is really helpful clarification. One followup: should I store this binary in PostGIS? I just want to avoid the situation where the server has to pull the entire binary file out of the database before querying it with PHP or Python. That would be a prohibitively slow step. –  dmahr Jan 14 '12 at 15:23
    
No, the file should be on the filesystem. No point in using a database. Even just opening a connection will probably take more time than the code above. –  johanvdw Jan 15 '12 at 20:44
    
Great, thanks a ton! This is very helpful. –  dmahr Jan 16 '12 at 14:31
add comment

This looks like three separate issues: one of infrastructure, one of architecture and one of event handling. I'll lay out one possible approach, but my answer is necessarily general.

Infrastructure

I recommend using a VPS hosting service like Linode (www.linode.com) for your server. This give you full (i.e., root) access to a professionally maintained server - no worries of the power going out or losing its connection to the Internet.

Architecture

There are so many options here that it can be really overwhelming. As an example, I run a couple of systems with GeoServer and OpenLayers. GeoServer is served up by Tomcat 7. The OpenLayers/jQuery front end is served up by Apache2. You can include Postgres/PostGIS for storing vector data, but this is not a good option for raster data. You could also set up a Python system using Django or even web.py (http://webpy.org/) for a rather simple controller. GeoServer lets you store raster data in the following formats:

  • ArcGrid - Arc Grid Coverage Format
  • GeoTIFF - Tagged Image File Format with Geographic information
  • Gtopo30 - Gtopo30 Coverage Format
  • ImageMosaic - Image mosaicking plugin
  • WorldImage - A raster file accompanied by a spatial data file

Event Handling

When the user clicks on the map you want to pop up a time-series plot of the field data at that point. First, set up a controller, which could be written in Python or Java, that listens for url requests with a latitude and longitude. This controller returns either a static image rendered on the server or json data that the client (jQuery) can turn into a plot.

Next, to get the XY data on the map you might use a function like this (see http://dev.openlayers.org/releases/OpenLayers-2.11/examples/click-handler.html):

onClick: function(evt) {
    var clickSpot = evt.xy;
    // pass clickSpot to the controller ...
}

I hope that helps.

share|improve this answer
    
Thanks for the writeup! Have you ever used any of those file formats to do three dimensional rasters that are over 8GB in size? I know that GeoTIFFs can't exceed 4GB for example. And will the server software be able to efficiently query a single z-axis time series of data? –  dmahr Jan 13 '12 at 12:22
    
Take a look at image pyramids. That may help with the file size issues. When handling the click event your code (controller) will receive the XY data and then search for and extract the z-value from each of the 300 timestamped images. That's a lot of processing and disk IO, so it could take a long time. If that "brute force" method takes too long then you could look into alternate algorithms and/or time-series storage schemes. –  katahdin Jan 13 '12 at 14:02
    
Yeah that "brute force" method will be slower than I'd prefer (it's even slow on a desktop computer). I guess I'm interested in framing the entire web mapping service around the best alternate storage scheme I can find. –  dmahr Jan 13 '12 at 21:23
add comment

If I understand correctly your question, I would create a time WMS service, for example with MapServer.

This way it will be easy to display the correct raster for each datetime (with a GetMap request), and request the values for a cell in a given datetime range (with a GetFeatureInfo request).

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.