//////////////////////////////////////////////////////////////////////////
//// This content is shared by all Elastic Beats. Make sure you keep the
//// descriptions here generic enough to work for all Beats that include
//// this file. When using cross references, make sure that the cross
//// references resolve correctly for any files that include this one.
//// Use the appropriate variables defined in the index.asciidoc file to
//// resolve Beat names: beatname_uc and beatname_lc
//// Use the following include to pull this content into a doc file:
//// include::../../libbeat/docs/loggingconfig.asciidoc[]
//// Make sure this content appears below a level 2 heading.
//////////////////////////////////////////////////////////////////////////

[[configuration-logging]]
== Configure logging

++++
<titleabbrev>Logging</titleabbrev>
++++

The `logging` section of the +{beatname_lc}.yml+ config file contains options
for configuring the logging output.
ifndef::serverless[]
The logging system can write logs to the syslog or rotate log files. If logging
is not explicitly configured the file output is used.

ifndef::win_only[]
["source","yaml",subs="attributes"]
----
logging.level: info
logging.to_files: true
logging.files:
  path: /var/log/{beatname_lc}
  name: {beatname_lc}
  keepfiles: 7
  permissions: 0644
----
endif::win_only[]

ifdef::win_only[]
["source","yaml",subs="attributes"]
----
logging.level: info
logging.to_files: true
logging.files:
  path: C:{backslash}ProgramData{backslash}{beatname_lc}{backslash}Logs
  name: {beatname_lc}
  keepfiles: 7
  permissions: 0644
----
endif::win_only[]

TIP: In addition to setting logging options in the config file, you can modify
the logging output configuration from the command line. See
<<command-line-options>>.

ifndef::win_only[]
WARNING: When {beatname_uc} is running on a Linux system with systemd, it uses
by default the `-e` command line option, that makes it write all the logging output
to stderr so it can be captured by journald. Other outputs are disabled. See
<<running-with-systemd>> to know more and learn how to change this.
endif::win_only[]
endif::serverless[]

ifdef::serverless[]
For example, the following options configure {beatname_uc} to log all the debug
messages related to event publishing:

["source","yaml",subs="attributes"]
----
logging.level: debug
logging.selectors: ["publisher"]
----

The logs generated by {beatname_uc} are written to the CloudWatch log group for
the function running on Amazon Web Services (AWS). To view the logs, go to the
the monitoring area of the AWS Lambda console and view the CloudWatch log group
for the function.

// TODO: When we add support for other cloud providers, we will need to modify
// this statement and possibly have a different attribute for each provider to
// show the correct text.
endif::serverless[]

[float]
=== Configuration options

You can specify the following options in the `logging` section of the
+{beatname_lc}.yml+ config file:

ifndef::serverless[]
[float]
==== `logging.to_stderr`

When true, writes all logging output to standard error output. This is
equivalent to using the `-e` command line option.

[float]
==== `logging.to_syslog`

When true, writes all logging output to the syslog.

NOTE: This option is not supported on Windows.

[float]
==== `logging.to_eventlog`

When true, writes all logging output to the Windows Event Log.

[float]
==== `logging.to_files`

When true, writes all logging output to files. The log files are automatically
rotated when the log file size limit is reached.

NOTE: {beatname_uc} only creates a log file if there is logging output. For
example, if you set the log <<level,`level`>> to `error` and there are no
errors, there will be no log file in the directory specified for logs.
endif::serverless[]

[float]
[[level]]
==== `logging.level`

Minimum log level. One of `debug`, `info`, `warning`, or `error`. The default
log level is `info`.

`debug`:: Logs debug messages, including a detailed printout of all events
flushed. Also logs informational messages, warnings, errors, and
critical errors. When the log level is `debug`, you can specify a list of
<<selectors,`selectors`>> to display debug messages for specific components. If
no selectors are specified, the `*` selector is used to display debug messages
for all components.

`info`:: Logs informational messages, including the number of events that are
published. Also logs any warnings, errors, or critical errors.

`warning`:: Logs warnings, errors, and critical errors.

`error`:: Logs errors and critical errors.

[float]
[[selectors]]
==== `logging.selectors`

The list of debugging-only selector tags used by different {beatname_uc} components.
Use `*` to enable debug output for all components. Use `publisher` to display
debug messages related to event publishing.

[TIP]
=====
The list of available selectors may change between releases, so avoid creating
tests that depend on specific selectors.

To see which selectors are available, run {beatname_uc} in debug mode
(set `logging.level: debug` in the configuration). The selector name appears
after the log level and is enclosed in brackets.
=====

To configure multiple selectors, use the following {beats-ref}/config-file-format.html[YAML list syntax]:
["source","yaml",subs="attributes"]
----
logging.selectors: [ harvester, input ]
----

ifndef::serverless[]
To override selectors at the command line, use the `-d` global flag (`-d` also
sets the debug log level). For more information, see <<command-line-options>>.
endif::serverless[]

[float]
==== `logging.metrics.enabled`

By default, {beatname_uc} periodically logs its internal metrics that have
changed in the last period. For each metric that changed, the delta from the
value at the beginning of the period is logged. Also, the total values for all
non-zero internal metrics are logged on shutdown. Set this to false to disable
this behavior. The default is true.

Here is an example log line:

[source,shell]
----------------------------------------------------------------------------------------------------------------------------------------------------
2017-12-17T19:17:42.667-0500    INFO    [metrics]       log/log.go:110  Non-zero metrics in the last 30s: beat.info.uptime.ms=30004 beat.memstats.gc_next=5046416
----------------------------------------------------------------------------------------------------------------------------------------------------

Note that we currently offer no backwards compatible guarantees for the internal
metrics and for this reason they are also not documented.

[float]
==== `logging.metrics.period`

The period after which to log the internal metrics. The default is 30s.

[float]
==== `logging.metrics.namespaces`

A list of metrics namespaces to report in the logs. Defaults to `[stats]`.
`stats` contains general Beat metrics. `dataset` may be present in some
Beats and contains module or input metrics.

ifndef::serverless[]
[float]
==== `logging.files.path`

The directory that log files are written to. The default is the logs path. See
the <<directory-layout>> section for details.

[float]
==== `logging.files.name`

The name of the file that logs are written to. The default is '{beatname_lc}'.

[float]
==== `logging.files.rotateeverybytes`

The maximum size of a log file. If the limit is reached, a new log file is
generated. The default size limit is 10485760 (10 MB).

[float]
==== `logging.files.keepfiles`

The number of most recent rotated log files to keep on disk. Older files are
deleted during log rotation. The default value is 7. The `keepfiles` options has
to be in the range of 2 to 1024 files.

[float]
==== `logging.files.permissions`

The permissions mask to apply when rotating log files. The default value is
0600. The `permissions` option must be a valid Unix-style file permissions mask
expressed in octal notation. In Go, numbers in octal notation must start with
'0'.

The most permissive mask allowed is 0640. If a higher permissions mask is
specified via this setting, it will be subject to an umask of 0027.

This option is not supported on Windows.

Examples:

* 0644: give read and write access to the file owner, and read access to all others.
* 0600: give read and write access to the file owner, and no access to all others.
* 0664: give read and write access to the file owner and members of the group
associated with the file, as well as read access to all other users.

[float]
==== `logging.files.interval`

Enable log file rotation on time intervals in addition to size-based rotation.
Intervals must be at least 1s. Values of 1m, 1h, 24h, 7*24h, 30*24h, and 365*24h
are boundary-aligned with minutes, hours, days, weeks, months, and years as
reported by the local system clock. All other intervals are calculated from the
unix epoch. Defaults to disabled.

[float]
==== `logging.files.suffix`

When a log rotation happens it can either rename older files with
an incresing index if `count` is configured. The other option is `date`
that appends the current date and time to the end of the filename.
When the log is rotated a new file is created and older files
remain untouched.
endif::serverless[]

[float]
==== `logging.files.rotateonstartup`

If the log file already exists on startup, immediately rotate it and start
writing to a new file instead of appending to the existing one. Defaults to
true.

[float]
==== `logging.json`

When true, logs messages in JSON format. The default is false.

deprecated::[7.16.0]

[float]
==== `logging.ecs`

When true, logs messages with minimal required Elastic Common Schema (ECS)
information.

deprecated::[7.16.0]

ifndef::serverless[]
[float]
==== `logging.files.redirect_stderr` experimental[]

When true, diagnostic messages printed to {beatname_uc}'s standard error output
will also be logged to the log file. This can be helpful in situations were
{beatname_uc} terminates unexpectedly because an error has been detected by
Go's runtime but diagnostic information is not present in the log file.
This feature is only available when logging to files (`logging.to_files` is true).
Disabled by default.
endif::serverless[]

[float]
=== Logging format

The logging format is generally the same for each logging output. The one
exception is with the syslog output where the timestamp is not included in the
message because syslog adds its own timestamp.

Each log message consists of the following parts:

* Timestamp in ISO8601 format
* Level
* Logger name contained in brackets (Optional)
* File name and line number of the caller
* Message
* Structured data encoded in JSON (Optional)

Below are some samples:

`2017-12-17T18:54:16.241-0500	INFO	logp/core_test.go:13	unnamed global logger`

`2017-12-17T18:54:16.242-0500	INFO	[example]	logp/core_test.go:16	some message`

`2017-12-17T18:54:16.242-0500	INFO	[example]	logp/core_test.go:19	some message	{"x": 1}`
