3

I have noticed that between 1-3% of SRTM .hgt files downloaded for a continent when un-zipped are of TGA image, font image or text format and do not contain elevation data.

Results are the same whether files are downloaded from NASA, VFP or mirror sites. I do not think it is my un-zipper as results are consistent. I am trying to track down what the contours look like, bit of "needle in a haystack", tho.

My concern of course is that these are virus or gov't tracking files. ;)

Any thoughts? Explanations?

Examples:

S35E147 - TGA image file
S36E117 - TeX font metrics file
N33E006 - text file

  • If you are using FTP, are you sure you're downloading in binary format? – BradHards Dec 11 '14 at 22:28
2

I can reproduce TeX and TGA type displays, but not the text file, in Nautilus file manager on ubuntu 12.04 LTS:

Sreenshot ubuntu Nautilus

SRTM3 v2.1 downloads from USGS: S35E147.hgt.zip, S36E117.hgt.zip, N33E006.hgt.zip

I think there is nothing wrong with the files themselves. It's probably the way Linux is detecting file types that goes wrong in some cases here.

HGT (stands for height) is a very simple format, see SRTM_Topo.pdf:

The DEM is provided as 16-bit signed integer data in a simple binary raster. There are no header or trailer bytes embedded in the file.

When an extension (.hgt) is not known, file type detection tries to look for "magic numbers", some unique number sequence identifying a format in the first few bytes of a file. As HGT files immediately start with the actual data, there might be rare cases where the data randomly matches the magic number of some arbitrary format.

For Ubuntu Nautilus mime type and magic number definitions can be found in
/usr/share/mime/packages/freedesktop.org.xml 7 (excerpts below).

S36E117.hgt - TeX font metrics

hexdump plus octal representation:

$ hd -b -n 16 S36E117.hgt
00000000  00 13 00 12 00 12 00 12  00 15 00 15 00 15 00 16  |................|
0000000 000 023 000 022 000 022 000 022 000 025 000 025 000 025 000 026

"TeX font metrics" defininion (excerpt):

<mime-type type="application/x-font-tex-tfm">
  <comment>TeX font metrics</comment>
  ...
  <magic priority="50">
    <match value="\000\021" type="string" offset="2"/>
    <match value="\000\022" type="string" offset="2"/>
  </magic>
</mime-type>

Octal bytes 000 022 at offset 2 seem to match "\000\022" (assuming 0n is octal and \ escapes a number in the string type).

S35E147 - TGA image

$ hd -n 16 S35E147.hgt
00000000  01 01 01 02 01 01 01 01  01 02 01 03 01 03 01 00  |................|

"TGA image" defininion (excerpt):

<mime-type type="image/x-tga">
  <comment>TGA image</comment>
  ...
  <magic priority="10">
    <match value="\1\1" type="string" offset="1"/>
    ...
  </magic>
</mime-type>

Bytes 01 01 at offset 1 seem to match values "\1\1".

N33E006 - text?

$ hd -v -n 64 N33E006.hgt
00000000  ff e0 ff e0 ff e0 ff e0  ff e0 ff e0 ff e0 ff e0  |................|
00000010  ff e0 ff e0 ff e0 ff e0  ff e0 ff e0 ff e0 ff e0  |................|
00000020  ff e0 ff e0 ff e0 ff e0  ff e0 ff e0 ff e0 ff e0  |................|
00000030  ff e0 ff e0 ff e0 ff e0  ff e0 ff e0 ff e0 ff e0  |................|

Hex dump for N33E006 starts with a sequence of "ff e0" which is signed 16 bit representation for "-32". Decimals 255 and 224 for "ff e0" are not really printable ASCII characters between 32 and 126, but might be seen as extended ASCII/8-bit encoding and there are no control characters (0-31), which might qualify as text?

5

You might try the USGS SRTM FTP site. I can assure you that each of the tiles you list as examples are there in their proper .hgt format and there are no suspicious files in the associated zip file.

enter image description here

And here is the tile S35E147, buffered with the surrounding eight tiles as well:

enter image description here

  • @pitney You're very welcome and I hope it helped. – WhiteboxDev Dec 11 '14 at 17:13
  • WhiteboxDev - Thank you for your reply. I had previously downloaded the files from the site you suggested and downloaded S35 again this morning and received the same (tga) file description. I made a map which displays fine, unfortunately this file is in an area of low relief so it is difficult to determine if contours are a problem. I will try to contact the Linux Dolphin file manager team and see if they can shed any light. – pitney Dec 11 '14 at 19:37
  • 1
    @pitney This is very odd because when I download the zip files from the USGS SRTM FTP site and unzip the tiles there are no other files contained in the directory other than the hgt file, which contains the raw elevation data. I'm not sure how you are getting a tga image file in there. Could there be something wrong with your unzipper? You might give this a try: whiteboxgeospatial.wordpress.com/2014/05/11/… – WhiteboxDev Dec 11 '14 at 19:42
1

The Dolphin file team? also believes that the file manager is detecting the wrong file type based on "magic numbers", see:

https://forum.kde.org/viewtopic.php?f=224&t=124038&p=325861#p325861

Again, thank you looking into this and hopefully it will provide an answer for others.
pitney

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.