Bad SRTM files?

Question

I have noticed that between 1-3% of SRTM .hgt files downloaded for a continent when un-zipped are of TGA image, font image or text format and do not contain elevation data.

Results are the same whether files are downloaded from NASA, VFP or mirror sites. I do not think it is my un-zipper as results are consistent. I am trying to track down what the contours look like, bit of "needle in a haystack", tho.

My concern of course is that these are virus or gov't tracking files. ;)

Any thoughts? Explanations?

Examples:

S35E147 - TGA image file
S36E117 - TeX font metrics file
N33E006 - text file

Answer 1

You might try the USGS SRTM FTP site. I can assure you that each of the tiles you list as examples are there in their proper .hgt format and there are no suspicious files in the associated zip file.

And here is the tile S35E147, buffered with the surrounding eight tiles as well:

Answer 2

I can reproduce TeX and TGA type displays, but not the text file, in Nautilus file manager on ubuntu 12.04 LTS:

SRTM3 v2.1 downloads from USGS: S35E147.hgt.zip, S36E117.hgt.zip, N33E006.hgt.zip

I think there is nothing wrong with the files themselves. It's probably the way Linux is detecting file types that goes wrong in some cases here.

HGT (stands for height) is a very simple format, see SRTM_Topo.pdf:

The DEM is provided as 16-bit signed integer data in a simple binary raster. There are no header or trailer bytes embedded in the file.

When an extension (.hgt) is not known, file type detection tries to look for "magic numbers", some unique number sequence identifying a format in the first few bytes of a file. As HGT files immediately start with the actual data, there might be rare cases where the data randomly matches the magic number of some arbitrary format.

For Ubuntu Nautilus mime type and magic number definitions can be found in
/usr/share/mime/packages/freedesktop.org.xml 7 (excerpts below).

S36E117.hgt - TeX font metrics

hexdump plus octal representation:

$ hd -b -n 16 S36E117.hgt
00000000  00 13 00 12 00 12 00 12  00 15 00 15 00 15 00 16  |................|
0000000 000 023 000 022 000 022 000 022 000 025 000 025 000 025 000 026

"TeX font metrics" defininion (excerpt):

<mime-type type="application/x-font-tex-tfm">
  <comment>TeX font metrics</comment>
  ...
  <magic priority="50">
    <match value="\000\021" type="string" offset="2"/>
    <match value="\000\022" type="string" offset="2"/>
  </magic>
</mime-type>

Octal bytes 000 022 at offset 2 seem to match "\000\022" (assuming 0n is octal and \ escapes a number in the string type).

S35E147 - TGA image

$ hd -n 16 S35E147.hgt
00000000  01 01 01 02 01 01 01 01  01 02 01 03 01 03 01 00  |................|

"TGA image" defininion (excerpt):

<mime-type type="image/x-tga">
  <comment>TGA image</comment>
  ...
  <magic priority="10">
    <match value="\1\1" type="string" offset="1"/>
    ...
  </magic>
</mime-type>

Bytes 01 01 at offset 1 seem to match values "\1\1".

N33E006 - text?

$ hd -v -n 64 N33E006.hgt
00000000  ff e0 ff e0 ff e0 ff e0  ff e0 ff e0 ff e0 ff e0  |................|
00000010  ff e0 ff e0 ff e0 ff e0  ff e0 ff e0 ff e0 ff e0  |................|
00000020  ff e0 ff e0 ff e0 ff e0  ff e0 ff e0 ff e0 ff e0  |................|
00000030  ff e0 ff e0 ff e0 ff e0  ff e0 ff e0 ff e0 ff e0  |................|

Hex dump for N33E006 starts with a sequence of "ff e0" which is signed 16 bit representation for "-32". Decimals 255 and 224 for "ff e0" are not really printable ASCII characters between 32 and 126, but might be seen as extended ASCII/8-bit encoding and there are no control characters (0-31), which might qualify as text?

Answer 3

The Dolphin file team? also believes that the file manager is detecting the wrong file type based on "magic numbers", see:

https://forum.kde.org/viewtopic.php?f=224&t=124038&p=325861#p325861

Again, thank you looking into this and hopefully it will provide an answer for others.
pitney