2

Motivation

I have satellite .tiff data files. I want to understand what information exists in these files. When I run gdalinfo on them, with the -json flag, I don't get any null values from the output, for missing attributes (i.e., attributes gdalinfo tried to find, but did not find). This is problematic because I want to determine what is missing, and I don't want different outputs from different files, as this does not allow me to report what is missing.

Target

I want to find a way to get the searched but not found attributes from gdalinfo.

Steps taken

Step 1: gdalinfo docs

My first step was the official docs for gdalinfo (see link [1]). I found found no ability to add a flag to get this data.

Step 2: Schema

Then I looked for a schema. I found two schemas. The first was a general schema for json objects returned by gdal (see link [2]). This is not specific enough for me. The second was from RFC44 (see link [3]). Unfortunately, this schema is outdated. So both do not allow me to know or infer what is missing.

Step 3: Source code

3.1 GDALInfo: From /master/apps/gdalinfo_lib.cpp

My next step was going to the source code to see if this data exists there. I found the implementation for gdalinfo in github (see link [4]), lines 240 to 1734, where the function is called GDALInfo.

The output of this function is a string. The string is the result of a serialization process on data that is collected in the following sections, described as follows, with starting line:

  1. Report general info: line 263
  2. Report projection: line 373
  3. Report Geotransform: line 504
  4. Report GCPs: line 562
  5. Report metadata: line 671
  6. Setup projected to lat/long transform if appropriate: line 695
  7. Report corners: line 741 Loop over bands: line 818

Each each of these sections orchestrates a search for different data on the source information. This search is encapsulated in an object. When target data is not found, a NULL pointer is kept as a value to an attribute/field of the encapsulating object.

Before the GDALInfo function terminates, in lines 1714-1733, we have:

    if (bJson)
    {
        json_object_object_add(poJsonObject, "bands", poBands);
        json_object_object_add(poStac, "raster:bands", poStacRasterBands);
        json_object_object_add(poStac, "eo:bands", poStacEOBands);
        json_object_object_add(poJsonObject, "stac", poStac);
        Concat(osStr, psOptions->bStdoutOutput, "%s",
               json_object_to_json_string_ext(
                   poJsonObject, JSON_C_TO_STRING_PRETTY
#ifdef JSON_C_TO_STRING_NOSLASHESCAPE
                                     | JSON_C_TO_STRING_NOSLASHESCAPE
#endif
                   ));
        json_object_put(poJsonObject);
    }

    if (psOptionsToFree != nullptr)
        GDALInfoOptionsFree(psOptionsToFree);

    return VSI_STRDUP_VERBOSE(osStr);

Here, given that the json flag is set (which makes bJson be true), the different objects are gathered and serialized to a final output string,osStr, which represents all the data gathered by the GDALInfo function. Since the data from these sections contains a NULL pointer for attributes gdalinfo did not find, this last serialization part is critical to determine if the user can get a representation of the fact that GDALInfo did not find something, such as a "null" string, appearing in the final output string, osStr.

3.2 Serializations: From /master/ogr/ogrsf_frmts/geojson/libjson/json_object.c

From what I can understand, the implementation for all the different serializations relies on a single C code implementation, located in lines 504-546 on the following file (see link [5]):

static int json_object_object_to_json_string(struct json_object *jso, struct printbuf *pb,
                                             int level, int flags)
{
    int had_children = 0;
    struct json_object_iter iter;

    printbuf_strappend(pb, "{" /*}*/);
    if (flags & JSON_C_TO_STRING_PRETTY)
        printbuf_strappend(pb, "\n");
    json_object_object_foreachC(jso, iter)
    {
        if (had_children)
        {
            printbuf_strappend(pb, ",");
            if (flags & JSON_C_TO_STRING_PRETTY)
                printbuf_strappend(pb, "\n");
        }
        had_children = 1;
        if (flags & JSON_C_TO_STRING_SPACED && !(flags & JSON_C_TO_STRING_PRETTY))
            printbuf_strappend(pb, " ");
        indent(pb, level + 1, flags);
        printbuf_strappend(pb, "\"");
        json_escape_str(pb, iter.key, strlen(iter.key), flags);
        if (flags & JSON_C_TO_STRING_SPACED)
            printbuf_strappend(pb, "\": ");
        else
            printbuf_strappend(pb, "\":");
        if (iter.val == NULL)
            printbuf_strappend(pb, "null");
        else if (iter.val->_to_json_string(iter.val, pb, level + 1, flags) < 0)
            return -1;
    }
    if (flags & JSON_C_TO_STRING_PRETTY)
    {
        if (had_children)
            printbuf_strappend(pb, "\n");
        indent(pb, level, flags);
    }
    if (flags & JSON_C_TO_STRING_SPACED && !(flags & JSON_C_TO_STRING_PRETTY))
        return printbuf_strappend(pb, /*{*/ " }");
    else
        return printbuf_strappend(pb, /*{*/ "}");
}

If we focus on lines 531-532, we see:

    if (iter.val == NULL)
            printbuf_strappend(pb, "null");

This means that null pointers are not thrown away, but are represented in the final output using a "null" string.

3.3 Conclusion

So, from what I understand:

  1. In every section, gdalinfo collects all searched data, whether found or not, through appending NULL pointers to dedicated fields in dedicated section spefic objects.
  2. The final portion of the function serializes the objects to a string. NULL pointers are translated to a "null" string, suggesting that the fact that data is not found is kept in the system.
  3. This output string, osStr should contain a "null" string for data the GDALInfo function searched for but could not find.

So under the hood, gdalinfo seems to have the necessary information to give me what I want in the terminal, but I am still not getting it - Why? Maybe it's a configuration option?

Step 4: Configuration options

Looking at the gdal configuration option in the official docs (see link [6]), I could not find anything that mentions "null" strings.

Current status

Having done the above, I still don't know how to reach my target. It seems gdalinfo has the necessary data to fetch searched but not found attributes, but I don't know why it is not present in any output.

So I'm still in the same spot: I want to be able to know, using the gdalinfo command on target files, which data is searched but not found.

More ideas

  1. Maybe I am expecting too much of gdalinfo, and I should look for other components in reading files, such as the driver. The driver gdal uses to read/write .tiff files is GTiff (see link [7]). This driver aheres to the GeoTIFF standard. Is it possible to infer a schema from the standard (see link [8])?

Links:

  1. gdalinfo docs: https://gdal.org/programs/gdalinfo.html
  2. RFC44 schema: https://github.com/OSGeo/gdal/blob/master/apps/data/ogrinfo_output.schema.json
  3. General json object schema: https://gdal.org/development/rfc/rfc44_gdalinfoxml.html
  4. GDALInfo implementation: https://github.com/OSGeo/gdal/blob/master/apps/gdalinfo_lib.cpp
  5. Serializations for json objects: https://github.com/OSGeo/gdal/blob/master/ogr/ogrsf_frmts/geojson/libjson/json_object.c
  6. gdal configuration docs: https://gdal.org/user/configoptions.html
  7. GTiff driver: https://gdal.org/drivers/raster/gtiff.html#raster-gtiff
  8. GeoTiff standard: https://www.ogc.org/standard/geotiff/
7
  • Please add also a link to test data.
    – user30184
    Commented Feb 9 at 17:10
  • My question is not data-specific, but function-specific. You can use any .tiff file you have, or see the example output in the gdalinfo docs. Commented Feb 9 at 17:40
  • Sorry, I was thinking about ogrinfo. What kind of attributes you are after? Do you see them in the normal text output of gdalinfo?
    – user30184
    Commented Feb 9 at 18:29
  • 1
    The gdalinfo json schema is probably this github.com/OSGeo/gdal/blob/…
    – user30184
    Commented Feb 9 at 18:35
  • Thank you!!!! How could I miss this?? How can I accept your answer? Commented Feb 9 at 18:42

1 Answer 1

3

There is actually a json schema file for gdalinfo. Initially it was added more than a year ago due to this feature request https://github.com/OSGeo/gdal/issues/6850.

The file has been moved into another location three weeks ago with this commit https://github.com/OSGeo/gdal/commit/392b8612c8c226621b35abb2cec1f7701cd152e6. The current location in GitHub is https://github.com/OSGeo/gdal/blob/721c775d38b71087d34fd0a1e63de06220e4fd68/apps/data/gdalinfo_output.schema.json

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.