2

I have to compute sea wave forecast for large period of time. Land information is of no use here.

Is there any file format that allows to not store no-data/fill values for land area to reduce file size? Chosen format for now is NetCDF, but it can be changed.

Other idea is to store only sea point values in binary file and write or read it using land sea mask. But it would probably have to be some custom data format.

Is it good or common practice to not include no-data points information in file?

Improve this question

1 Answer 1

Reset to default
4

NetCDF can use compression, like a zip file, in order to decrease the storage size of redundant information. So if you have a global raster and all the land values are set to -9999 then the compression algorithm won't simply store a zillion -9999s, it will do something like "row 23 is 4.1,3.2,4.5,-9999 x 1000, 2.3, 4.5" - in those quote marks I've described 1005 values in about 30 characters. That's what compression algorithms win for you (only more sophisticated and better than my illustrative example).

Alternatively instead of a raster you could store your data as a set of (X,Y,Z) point triples for each of your data locations. This can end up bigger than a raster since a raster can be described with two vectors (of X and Y locations) and a matrix of Z values. But the rest of your analysis and mapping might be raster-oriented and so you'd be forever converting back and forth.

Improve this answer
2
  • I've checked NetCDF internal compression with different compression levels. For my ~40MB .nc files it gives similar results to standard gzip compression - result is ~7.5MB. That's an option to consider.
    – kazik
    Commented Jun 6, 2019 at 10:25
  • I was also wondering, if there is some feature, maybe in other formats, like hdf, geotiff, that allows You to specify land sea mask and the file doesn't store information that You don't need, but it returns some nodata value if You ask for this point.
    – kazik
    Commented Jun 6, 2019 at 10:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.