parse_date throws exception on error

[jalogisch]

When using the parse_date function the rule will exit with an exception if the parse_date does not work.

Backround

parse_date could be used in the when part of the rules to check what date format is used and then act on that.

rule "correct time"
when
   parse_date(value:$message.log_date, pattern:"MMM dd HH:mm:ss.SSS", timezone:"CET");
then
    let date = now("CET");
    let new_date = concat(to_string(date.year), to_string($message.log_date));
    let time = parse_date(value:new_date, pattern:"yyyyMMM dd HH:mm:ss.SSS", timezone:"CET");
    set_field("timestamp",time);
end

Additional it is not possible to have multiple parse_date rules in the same rule to catch all possible timestamps with multiple callings of parse_date.

let new_date = to_string($message.log_date);
let time = parse_date(value:new_date, pattern:"MMM dd HH:mm:ss.SSS", timezone:"CET");
let time = parse_date(value:new_date, pattern:"MMM dd HH:mm:ss ZZZ", timezone:"CET");
let time = parse_date(value:new_date, pattern:"MMM dd HH:mm:ss.SSS ZZZ", timezone:"CET");
set_field("processed_time", time);

The above is possible with other functions (like grok).

Environment

• Graylog Version: 2.4.3

[joschi]

when
   parse_date(value:$message.log_date, pattern:"MMM dd HH:mm:ss.SSS", timezone:"CET");
then
  // ...
end

Apart from the semicolon in the when block which syntactically isn't correct, what should that rule do?

[jalogisch]

The rule would be the option to manipulate the date - if a specific date string is found.

In this case add the current year to the message because that is missing in the provided timestamp.

This is needed because of vendor specific bullshit (in this example Cisco Timestamps).

Six possible timestamps from the same DeviceFamily depending on the Software Level need to be normalized.

    # IOS
    "MMM dd HH:mm:ss.SSS ZZZ",
    "MMM dd HH:mm:ss ZZZ",
    "MMM dd HH:mm:ss.SSS",
    
    # Nexus
    "YYYY MMM dd HH:mm:ss.SSS ZZZ",
    "YYYY MMM dd HH:mm:ss ZZZ",
    "YYYY MMM dd HH:mm:ss.SSS",

Cisco has a bug open for that - but it is not that important for them (for a few years) to fix that.

[joschi]

The rule would be the option to manipulate the date - if a specific date string is found.

The aspect of user experience aside, why can't you use a Grok pattern or regular expression for that?

[jalogisch]

Most important here is that the parse_date throws an expection if it fails to run.

So you are not able, like with a Grok rule check if the pattern would match or not - or give multiple pattern and one of them will match like #244

when
    // make 'Feb  5 13:33:22.111' a ISO8601
    // cisco did not use 05 but <space>5 for days with a single digit
    has_field("log_date") AND
    grok(pattern: "%{MONTH}  %{MONTHDAY} %{TIME}", value:to_string($message.log_date)).matches == true

I didn't know if the parse date or regex would be faster - but it would be for some users more intuitive to check if a date pattern matches or not.