I've got a large time-series of rasters, derived from satellite images. Each raster is about 2000 x 2000 pixels, and I've got about 10,000 individual images. I'm trying to work out what the best way to store these is.
Of course, the best way to store them will depend on what I want to do with them - and in this case I want to be able to look at a time series for individual pixels, time-series averaged over specific areas (eg. vector administrative area data overlain on the raster), produce average images over various timescales, and view individual images.
The approach I've used before for shorter time series has just been to stack all of the images into a multi-band GeoTIFF - but I'm not even sure if a GeoTIFF will allow you to have 10,000 bands, or how well it will perform. An alternative would be to store them in thousands of separate files, but that would probably require a lot of custom programming work to be able to do the analysis.
I'm sure I can't be the first person to have this problem - what approaches are recommended for dealing with this sort of volume of time-series data?
Ideally I'd prefer solutions that work nicely with Python and don't require ArcGIS - but I'd be interested in any sensible ways forward.