NMEA to CSV with timestamp using gpsbabel

Question

I tried to convert a file in NMEA to csv using gpsbabel,

$ gpsbabel -i nmea -f GPS_20160204_092027.log -o csv -F GPS_20160204_092027.csv

I get a csv file with only latitude and longitude, but was hoping to get many other information stored in the original .log file, such as the timestamp.

What can I do to include other information (like the timestamp) as well in the converted csv file?

Edit: Here's the first 11 lines of my sample data, GPS_20160204_092027.log

$ADVER,3080,2.2
$GPRMC,092027.696,A,1438.9787,N,12104.3738,E,1.32,347.54,040216,,,A*6F
$GPGGA,092028.696,1438.9795,N,12104.3782,E,1,04,1.7,121.0,M,39.1,M,,0000*5B
$GPGSA,A,3,31,14,10,22,,,,,,,,,3.5,1.7,3.1*36
$GPRMC,092028.696,A,1438.9795,N,12104.3782,E,1.68,64.76,040216,,,A*5F
$GPVTG,64.76,T,,M,1.68,N,3.1,K,A*33
$GPGGA,092029.696,1438.9834,N,12104.3702,E,1,04,1.7,141.6,M,39.1,M,,0000*56
$GPGSA,A,3,31,14,10,22,,,,,,,,,3.5,1.7,3.1*36
$GPGSV,3,1,11,14,58,103,35,22,41,224,31,31,38,007,30,10,19,175,26*77
$GPGSV,3,2,11,25,00,039,,26,65,298,,16,46,247,33,29,25,059,*75
$GPGSV,3,3,11,27,13,195,,03,11,306,,21,03,127,*41

Answer 1

Thanks for your update with some sample data. As one can see from your sample, the nmea format basically already is csv but values might occur in rather uncommon formats and different nmea sentences (the one starting with $) often store the same information simply in a different format until the next new piece of information occurs. Hence, it is most often enough to evaluate all lines of only one of the sentences that holds the information that you are interested in.

Since I don't know what exact information you need (some of the sentences store really complicated stuff regarding the GPS signal), in the following I went for values for date, time, lat, lon, speed. However, it should not be too difficult to adapt the example to something else, otherwise just ask.

See inline comments for further explanation. In order to run this script, save the code in a .py file, adapt the INPUT_FILENAME and OUTPUT_FILENAME variables at the top of it to your needs, then just run Python filename.py in a console.

import csv
from datetime import datetime
import math

# adapt this to your file
INPUT_FILENAME = 'in.nmea'
OUTPUT_FILENAME = 'out.csv'

# open the input file in read mode
with open(INPUT_FILENAME, 'r') as input_file:

    # open the output file in write mode
    with open(OUTPUT_FILENAME, 'wt') as output_file:

        # create a csv reader object from the input file (nmea files are basically csv)
        reader = csv.reader(input_file)

        # create a csv writer object for the output file
        writer = csv.writer(output_file, delimiter=',', lineterminator='\n')

        # write the header line to the csv file
        writer.writerow(['date_and_time', 'lat', 'lon', 'speed'])

        # iterate over all the rows in the nmea file
        for row in reader:

            # skip all lines that do not start with $GPRMC
            if not row[0].startswith('$GPRMC'):
                continue

            else:

                # for each row, fetch the values from the row's columns
                # columns that are not used contain technical GPS stuff that you are probably not interested in
                time = row[1]
                warning = row[2]
                lat = row[3]
                lat_direction = row[4]
                lon = row[5]
                lon_direction = row[6]
                speed = row[7]
                date =  row[9]

                # if the "warning" value is "V" (void), you may want to skip it since this is an indicator for an incomplete data row)
                if warning == 'V':
                    continue

                # merge the time and date columns into one Python datetime object (usually more convenient than having both separately)
                date_and_time = datetime.strptime(date + ' ' + time, '%d%m%y %H%M%S.%f')

                # convert the Python datetime into your preferred string format, see http://www.tutorialspoint.com/python/time_strftime.htm for futher possibilities
                date_and_time = date_and_time.strftime('%y-%m-%d %H:%M:%S.%f')[:-3] # [:-3] cuts off the last three characters (trailing zeros from the fractional seconds)

                # lat and lon values in the $GPRMC nmea sentences come in an rather uncommon format. for convenience, convert them into the commonly used decimal degree format which most applications can read.
                # the "high level" formula for conversion is: DDMM.MMMMM => DD + (YY.ZZZZ / 60), multiplicated with (-1) if direction is either South or West
                # the following reflects this formula in mathematical terms.
                # lat and lon have to be converted from string to float in order to do calculations with them.
                # you probably want the values rounded to 6 digits after the point for better readability.
                lat = round(math.floor(float(lat) / 100) + (float(lat) % 100) / 60, 6)
                if lat_direction == 'S':
                    lat = lat * -1

                lon = round(math.floor(float(lon) / 100) + (float(lon) % 100) / 60, 6)
                if lon_direction == 'W':
                    lon = lon * -1

                # speed is given in knots, you'll probably rather want it in km/h and rounded to full integer values.
                # speed has to be converted from string to float first in order to do calculations with it.
                # conversion to int is to get rid of the tailing ".0".
                speed = int(round(float(speed) * 1.852, 0))

                # write the calculated/formatted values of the row that we just read into the csv file
                writer.writerow([date_and_time, lat, lon, speed])

Running this code on your sample, it will produce a csv file with the content of

date_and_time,lat,lon,speed
16-02-04 09:20:27.696,14.649645,121.072897,2
16-02-04 09:20:28.696,14.649658,121.07297,3

Note that this is all there is, all other lines only store redundant information with regards to the columns that are analyzed.

Answer 2

The actual answer to the question asked is in the documentation of GPSBabel's CSV writer at https://www.gpsbabel.org/htmldoc-development/fmt_csv.html "... contains only a very minimal information. For general purpos use, you'll probably be happer with our universal csv (unicsv) format.}

gpsbabel -t -i nmea -f /tmp/xx -o unicsv -F -

Shows just about everything that GPSBabel can pull out of that file.

No,Latitude,Longitude,Altitude,Speed,Course,FIX,HDOP,VDOP,PDOP,Satellites,Date,Time 1,14.649645,121.072897,,0.68,347.5,,,,,,2016/02/04,04:20:27.696 2,14.649658,121.072970,121.0,0.86,64.8,"3d",1.70,3.10,3.50,4,2016/02/04,04:20:28.696 3,14.649723,121.072837,141.6,,,"3d",1.70,3.10,3.50,4,2016/02/04,04:20:29.696