Recipes

log4j

My personal favorite: Multiline for log4j

filter {
  multiline {
    type => "log4j"
    pattern => "^20"
    negate => true
    what => "previous"
  }
}

This is cheating which should last until 2099 as it looks for lines beginning with 20 (e.g. 2000 - 2099). If the line doesn't start with 20 it is part of a stack trace and is appended to the previous line.

Here are the associated grok lines for log4j (defaults): http://pastebin.com/n8DuaMeQ Sorry, they don't paste well here. Too many escapes.

ColdFusion Recipe

This recipe is a rough draft for a CF 8 environment, though I believe it works just fine for v9. Take note that there are two different log formats we need to deal with: cfserver.log, and the regular CFMX CSV/quoted format.

The format of ColdFusion log this was made to process is as follows:

"Severity","ThreadID","Date","Time","Application","Message"
ex: "Error","app-01","01/01/11","01:01:11","test01","loggity log log"

The multiline check is to verify that the line begins with a quotation mark, otherwise it will be treated as an appendage to the previous log entry.

In the CFSERVER format we are shooting to process things like:

10/04 01:01:01 Information [app-01] - Begin logomatic...
AND
2011-09-01 01:01:01.111 WARN little.app.that.logged:  Starting logging

The multiline check here is to verify the line begins with one of the two date formats listed above, otherwise it will be appended to the last seen log that did begin with a our timestamp.

An issue found with cfserver.log parsing is that it will use two different timestamp formats. This is the reason for CFSEVERTIME's OR logic.

I am using the monolithic JAR in this example, and we must specify a patterns_dir with our grok filters to include the custom patterns.

/etc/logstash/patterns/cfusion:

CFSERVERTIME_OTHER %{MONTHNUM:month}/%{MONTHDAY:day} %{TIME:time}
CFSERVERTIME %{TIMESTAMP_ISO8601:timestamp8601}|%{CFSERVERTIME_OTHER:timestamp}

CFUSION %{QS:severity},%{QS:threadid},"%{DATE_US:date}","%{TIME:time}",%{QS:application}?,%{QS:message}
CFSERVER %{CFSERVERTIME} %{WORD:severity} %{DATA:application} %{GREEDYDATA:message}

The field added below in the grok statement is so that we can avoid the quoted characters. This allows us to easily process a timestamp with the date filter.

/etc/logstash/collector.conf

  grok {
    type => "cfusion"
    patterns_dir => "/etc/logstash/patterns"
    pattern => "%{CFUSION}"
	add_field => [ "timestamp","%{date} %{time}" ]
  }

  multiline {
    type => "cfusion"
    pattern => "^\".*$"
    negate => true
    what => "previous"
  }
  date {
    type => "cfusion"
    timestamp => "MM/dd/yy HH:mm:ss"
  }
  grok {
    type => "cfserver"
    patterns_dir => "/etc/logstash/patterns"
    pattern => "%{CFSERVER}"
  }
  multiline {
    type => "cfserver"
    pattern => "^(([0-9]+-(?:0?[1-9]|1[0-2])-(?:3[01]|[1-2]?[0-9]|0?[1-9]))|((?:0?[1-9]|1[0-2])/(?:3[01]|[1-2]?[0-9]|0?[1-9]))).*$"
    negate => true
    what => "previous"
  }
  date {
    type => "cfserver"
    timestamp8601 => "yyyy-MM-dd HH:mm:ss.SSS"
    timestamp => "MM/dd HH:mm:ss"
  }

More to come!