Novell Home

Collector Development Topics

DATES AND TIMES

Most received event records will contain a timestamp that indicates when the event occurred. It's important to understand how dates and times are handled by Sentinel in order to properly parse out the timestamp from the record and construct the appropriate output Event. The Collector provides a number of facilities for handling these scenarios.

Handling Timestamps

First, some background:

  • The world has a wide variety of timezones, and not all of them are on 1-hour boundaries, follow daylight savings, etc.
  • Sentinel's underlying framework uses Java, which has a rich, complex date/time handling system that fully understands timezones and all the complexities thereof.
  • Collectors themselves use Javascript, which has more limited support for timezones (it supports only the "letter-code" timezones like 'EST', not the full descriptors like 'Americas/New_York').
  • The Collector includes the date.js library, which provides a large number of utility functions useful for manipulating dates.
  • The Collector also provides some additional automated features to properly handle dates within Sentinel.
  • Sentinel itself stores all times as UTC, so that it can properly sequence events from across the world in time.

Now let's look at how normal Collectors handle setting the event time. Imagine to start off with that you have an event source in New York, a Collector Manager in Berlin, the Sentinel server in Tokyo, and a security analyst connected to the Sentinel server in San Francisco.

OK, so the event source generates events. There are three possible scenarios:

  1. The event source tags the events with time in UTC (Windows does this, for example).
  2. The event source tags the events with a local timestamp that includes the timezone information (e.g. 12:34:21 EST|Americas/New_York)
  3. The event source tags the events with a local timestamp that does NOT include the timezone (this is generally a bad idea, but not uncommon - see syslog).

All these cases are handled easily by the Collector. Internally, JavaScript always works in local time (in this case, Berlin time), so it will assume any date you pass to it (e.g. to create a Date object) is in Berlin time. The code has to explicitly shift the timestamp to a different timezone. This is all handled in the Collector template to make things easier, essentially when the Event.send() method is called, the template will automatically do any timestamp shifting required to make things work correctly. The Collector should be written to do the following:

  • For now, ignore any timezone associated with the event and create a JS Date object with the time string input. The Collector will automatically assume that the timestamp is in the local time of the Collector Manager (Berlin, in this example) even if it's actually some other timezone or UTC. Then, depending on the scenarios above:
    1. If the observer reports in UTC regardless of what actual timezone it actually resides in, set the instance.CONFIG.params.reportsUTC parameter to true.
    2. If you get the timezone in the event, use that to set the rec.s_TimeZone variable.
    3. If you don't get the timezone in the event, edit the Event Source node in ESM and set the timezone there. That will be passed to the Collector as rec.s_Timezone.
  • Pass your Date object to Event.setObserverEventTime().

When the Sentinel Event is sent, the template will take your Date object (which is in local time per the Collector Manager), the rec.s_TimeZone variable (which should report the timezone of the observer), and the reportsUTC parameter setting and based on all these factors will:

  • Shift the Event timestamp from the Collector local time to UTC.
  • Set the ObserverTZ field (Sentinel 7+) from rec.s_TimeZone.
  • Calculate the rest of the ObserverTZ* fields based on the local time of the Observer.

Examples

For our example above, let's assume the observer is sending local time without a timestamp (typical of syslog, for example). The timestamp will therefore be New York local time.

The release code for the Collector should extract that timestamp and construct a Date object. Since the Date constructor recognizes several common date formats, the developer may just be able to pass the timestamp portion of the string to the constructor. If there's some chance of ambiguity (differing month/day orders for different locales, for example), the developer can re-order the fields or use the Date.parseExact() method with a format string. Since there's no timezone (and if there were, the developer would ignore it), the Collector will assume the timestamp is for local time and create a Date in Berlin local time with the time of the timestamp. This is obviously off by several hours.

Since the observer is sending local time but not providing the timezone, the implementor must manually configure the Event Source node with a timezone. This will appear in the input record as rec.s_TimeZone.

When the Event is sent via Event.send(), the template will see that the real observer timezone is Americas/New_York and will shift the timestamp from Berlin time to New York time. It will then shift the timestamp to UTC time as that is what Sentinel needs. It will set the ObserverTZ field to rec.s_TimeZone.

When the framework receives the event from the Collector, it will take the UTC timestamp and the timezone in ObserverTZ and calculate the other ObserverTZ* fields in the local time of the observer.

When the Sentinel server in Tokyo gets the Event, it will store in UTC. The local time of the Sentinel server is irrelevant.

When the security analyst in San Francisco looks at the data, she will see the data in her local time (set by the local machine and/or web browser), which is important so she can think "oh, that happened an hour ago" without performing mental gymnastics. If the analyst wants to think about when the event occurred with respect to the observer's local time, she can look at the ObserverTZ* fields. Correlation and other analytics will depend on having a consistent UTC date stored in the event to provide globally accurate event sequencing.

Collector Development Guide

Development Topics

© 2014 Novell