How can I convert a UNIX timestamp to a datetime format within the GeoEvent extension?

Question

I assume that this can only be done with the use of a Field Calculator processor. However, there is little documentation about how to insert functions and about date- and time transformation in particular.

At the moment, the data comes in like this:

"lastUpdated":148043564

It is a double datatype. With the use of a fieldmapper I extracted this field into a custom definition. The new field is of a double datatype too (too prevent problems). This works well, however, I would like to add a new field to the incoming stream of data. This new field must have a 'human-readable' datetime value, like e.g. YYYY/MM/DD HH/MM/SS. What function do I need to fill in to achieve this and how? I'm assuming that my new field has to be of a date character.

Answer 1

The GeoEvent team provided me some info that might help answer your issue:

At the moment, the data comes in like this:
"lastUpdated":148043564"

That’s an epoch long integer expressed in seconds (I assume). So first they need to specify the GeoEvent Definition used by their inbound connector treat this attribute field as a ‘Date’ (not a ‘Double’). If you bring the attribute value in as Double it will forever be Double (not a Date) and you’ll have a hard time changing it.

They could then develop a custom processor which leverages java.text.SimpleDateFormat to convert the Date to a String … but I’m assuming they want an out-of-the-box solution.

GeoEvent expects all date/time epochs to be expressed in milliseconds, not seconds. Lucky for us scaling the date/time value is pretty simple. Once you have 148043564 being handled as a Date you can configure a Field Calculator with an expression myDateTimeEpoch * 1000 and write the result back into the field myDateTimeEpoch. This is how you scale the date/time from seconds to milliseconds … 148043564 becomes 148043564000.

This trick only works because, under the hood, GeoEvent is handling Date types consistent with Java which represents them as long integer epoch values. This is also why I can add a “five hours of milliseconds” to a Date in order to artificially push the value from one time zone to another.

The only way (out-of-the-box) to convert a Date to a “human readable” String is to allow an outbound Text adaptor to create a representation of the date/time as a formatted string for you. That means you have to send the event record to an output – like a TCP/Text output.

If you need to apply additional real-time analytics on the event record before obtaining the String representation of the date/time, you have to do that before sending the event record to the output … or figure out how to “output” the event and then re-ingest it for additional processing by another GeoEvent Service (this can get rather messy).

The JSON outbound adapter maintains Date data as epoch long integers. So you must use an outbound connector which leverages the Text adaptor if you want to create a human readable value from a Date.