3

I have an application that creates ASCII grid files (Arc/ESRI-format). For the actual cell values, the quantities I am dealing with are sometimes very small, and so it writes these out in scientific E notation, e.g.: 9.99547007184078E-06

Will ArcMap successfully read these files without truncating the number, and understand the E notation? I suspect it is not interpreting the values correctly, but just reading the first few digits of the cell values - in the 'Table of Contents' the high and low values are listed as ranging from vastly different values than in the raw data. Querying the cell pixel values also gives different values than would be expected.

If not is there an alternative (GRASS GIS, QGIS etc) that understands this format?

Addition: here is the header part of the file in question

ncols         557
nrows         300
xllcorner     209428.33804321
yllcorner     89000
cellsize      10
NODATA_value  -9999

I am importing the ASCII files using ASCII to raster, and selecting 'FLOAT' as the output data type. The values are truncated to something like 9.9954 in the above case.

  • 3
    Please edit the question to provide the header of the ASCII file in question. What gives you this "feeling"? How many is a "few digits"? (32-bit float only uses 7 significant digits) – Vince Jan 6 '15 at 18:32
  • Should work, any reason to believe it doesn't? I know the GeoTools reader does. – Ian Turton Jan 6 '15 at 19:38
  • It is badly documented, though in docs.codehaus.org/display/GEOTOOLS/ArcInfo+ASCII+Grid+format: val(nox,noy) (f) = individual grid values, column varying fastest in integer format. Grid values are stored as integers but can be read as floating point values. And Wikipedia writes The remainder of the file lists the raster values for each cell, starting at the upper-left corner. These real numbers (with optional decimal point, if needed) are delimited using a single space character. – user30184 Jan 6 '15 at 20:13
  • This page, however, suggests that scientific notation is supported but with some caveats: code.env.duke.edu/projects/mget/export/HEAD/MGET/Trunk/…. For example: Values where the exponent is less than -38 (e.g. -39, -40, and so on) are converted to 0. – user30184 Jan 6 '15 at 20:21
  • The numbers of significant digits will indeed be reduced, for the simple reason that ESRI floating point rasters store their values as single-precision IEEE floats. As far as how ArcMap actually behaves, why not do a round trip: read a grid in to ArcMap, export it again, read that back into the original software, and compare the original with the new grid in the original software to see what ArcMap did to the data. That will show you whatever patterns might be occurring in the errors. If all fails, you can work around the problem by exporting two integer grids: an exponent and a mantissa. – whuber Jan 6 '15 at 22:26
3

There's probably multiple things going on to produce the behavior you are seeing.

First of all, when you create a FLOAT raster, you truncate the 64-bit floating-point values to 32-bit values, with a predictable loss of precision (based on the smaller mantissa -- from 53 to 24 binary digits).

Second, it's likely that you are looking at the default display format, which is not the same as the value.

To provide an example of the difference between FLOAT and DOUBLE values and their display representations, I whipped up a trivial 'C' app:

#include <stdio.h>
#include <stdlib.h>

int     main(void)
{
        double  d = 9.99547007184078E-06;
        float   f;
        int     i, max = 16;

        f = d;
        fprintf(stdout,"i\t    d\t\t\t    f\n");
        for (i = max-1; i > 1; i--) {
                fprintf(stdout,"%2d  %.*e%*s\t%.*e\n",i,i,d,max-i,"",i,f);
        }
        return EXIT_SUCCESS;
}

The output is as follows:

i           d                       f
15  9.995470071840780e-06       9.995470463763922e-06
14  9.99547007184078e-06        9.99547046376392e-06
13  9.9954700718408e-06         9.9954704637639e-06
12  9.995470071841e-06          9.995470463764e-06
11  9.99547007184e-06           9.99547046376e-06
10  9.9954700718e-06            9.9954704638e-06
 9  9.995470072e-06             9.995470464e-06
 8  9.99547007e-06              9.99547046e-06
 7  9.9954701e-06               9.9954705e-06
 6  9.995470e-06                9.995470e-06
 5  9.99547e-06                 9.99547e-06
 4  9.9955e-06                  9.9955e-06
 3  9.995e-06                   9.995e-06
 2  1.00e-05                    1.00e-05

You can clearly see that the double value d is different than the float value f after the eighth digit (seventh right of decimal) due to mantissa truncation, but once i drops to 6, the display values are the same. It's important to note that the values at these display resolutions are the same; only the display width changes.

The only way to be absolutely certain of discrete values in a floating-point representation would be to cast them to hexadecimal (e.g. 0x3727b238) or to multiply by a value (such as a large power of ten), and round to integer. This should of course be done at the highest acceptable precision, since the truncation would then be under your control.

If you need to preserve the 64-bit values, you'll need to use a 64-bit storage format, choosing "DOUBLE" instead of "FLOAT" for the output data type.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.