Skip to main content

parse_log

Parses common log formats into structured data. This is easier and often much faster than grok.

# Common config fields, showing default values
label: ""
parse_log:
  format: ""
  codec: ""

Fields​

format​

A common log format to parse.

Type: string
Default: ""
Options: syslog_rfc5424, syslog_rfc3164.

codec​

Specifies the structured format to parse a log into.

Type: string
Default: ""
Options: json.

best_effort​

Still returns partially parsed messages even if an error occurs.

Type: bool
Default: true

allow_rfc3339​

Also accept timestamps in rfc3339 format while parsing. Applicable to format syslog_rfc3164.

Type: bool
Default: true

default_year​

Sets the strategy used to set the year for rfc3164 timestamps. Applicable to format syslog_rfc3164. When set to current the current year will be set, when set to an integer that value will be used. Leave this field empty to not set a default year at all.

Type: string
Default: "current"

default_timezone​

Sets the strategy to decide the timezone for rfc3164 timestamps. Applicable to format syslog_rfc3164. This value should follow the time.LoadLocation format.

Type: string
Default: "UTC"

Codecs​

Currently the only supported structured data codec is json.

Formats​

syslog_rfc5424​

Attempts to parse a log following the Syslog rfc5424 spec. The resulting structured document may contain any of the following fields:

  • message (string)
  • timestamp (string, RFC3339)
  • facility (int)
  • severity (int)
  • priority (int)
  • version (int)
  • hostname (string)
  • procid (string)
  • appname (string)
  • msgid (string)
  • structureddata (object)

syslog_rfc3164​

Attempts to parse a log following the Syslog rfc3164 spec. The resulting structured document may contain any of the following fields:

  • message (string)
  • timestamp (string, RFC3339)
  • facility (int)
  • severity (int)
  • priority (int)
  • hostname (string)
  • procid (string)
  • appname (string)
  • msgid (string)