\newpage

# Features in detail

In this section we present several Barman features and discuss their
applicability and the configuration required to use them.

This list is not exhaustive, as many scenarios can be created working
on the Barman configuration. Nevertheless, it is useful to discuss
common patterns.

## Backup features

### Incremental backup

Barman implements **file-level incremental backup**. Incremental
backup is a type of full periodic backup which only saves data changes
from the latest full backup available in the catalog for a specific
PostgreSQL server. It must not be confused with differential backup,
which is implemented by _WAL continuous archiving_.

> **NOTE:** Block level incremental backup will be available in
> future versions.

> **IMPORTANT:** The `reuse_backup` option can't be used with the
> `postgres` backup method at this time.

The main goals of incremental backups in Barman are:

- Reduce the time taken for the full backup process
- Reduce the disk space occupied by several periodic backups (**data
  deduplication**)

This feature heavily relies on `rsync` and [hard links][8], which
must therefore be supported by both the underlying operating system
and the file system where the backup data resides.

The main concept is that a subsequent base backup will share those
files that have not changed since the previous backup, leading to
relevant savings in disk usage. This is particularly true of VLDB
contexts and of those databases containing a high percentage of
_read-only historical tables_.

Barman implements incremental backup through a global/server option
called `reuse_backup`, that transparently manages the `barman backup`
command. It accepts three values:

- `off`: standard full backup (default)
- `link`: incremental backup, by reusing the last backup for a server
  and creating a hard link of the unchanged files (for backup space
  and time reduction)
- `copy`: incremental backup, by reusing the last backup for a server
  and creating a copy of the unchanged files (just for backup time
  reduction)

The most common scenario is to set `reuse_backup` to `link`, as
follows:

``` ini
reuse_backup = link
```

Setting this at global level will automatically enable incremental
backup for all your servers.

As a final note, users can override the setting of the `reuse_backup`
option through the `--reuse-backup` runtime option for the `barman
backup` command. Similarly, the runtime option accepts three values:
`off`, `link` and `copy`. For example, you can run a one-off
incremental backup as follows:

``` bash
barman backup --reuse-backup=link <server_name>
```

### Limiting bandwidth usage

It is possible to limit the usage of I/O bandwidth through the
`bandwidth_limit` option (global/per server), by specifying the
maximum number of kilobytes per second. By default it is set to 0,
meaning no limit.

> **IMPORTANT:** the `bandwidth_limit` option is supported with the
> `postgres` backup method, but the `tablespace_bandwidth_limit` option
> is available only if you use `rsync`.

In case you have several tablespaces and you prefer to limit the I/O
workload of your backup procedures on one or more tablespaces, you can
use the `tablespace_bandwidth_limit` option (global/per server):

``` ini
tablespace_bandwidth_limit = tbname:bwlimit[, tbname:bwlimit, ...]
```

The option accepts a comma separated list of pairs made up of the
tablespace name and the bandwidth limit (in kilobytes per second).

When backing up a server, Barman will try and locate any existing
tablespace in the above option. If found, the specified bandwidth
limit will be enforced. If not, the default bandwidth limit for that
server will be applied.

### Network Compression

It is possible to reduce the size of transferred data using
compression. It can be enabled using the `network_compression` option
(global/per server):

> **IMPORTANT:** the `network_compression` option is not available
> with the `postgres` backup method.

``` ini
network_compression = true|false
```

Setting this option to `true` will enable data compression during
network transfers (for both backup and recovery). By default it is set
to `false`.

### Backup Compression

Barman can use the compression features of pg_basebackup in order to
compress the backup data during the backup process. This can be enabled
using the `backup_compression` config option (global/per server):

> **IMPORTANT:** the `backup_compression` and other options discussed
> in this section are not available with the `rsync` or `local-rsync`
> backup methods. Only with `postgres` backup method.

#### Compression algorithms

Setting this option will cause pg_basebackup to compress the backup
using the specified compression algorithm. Currently, supported
algorithm in Barman are: `gzip`, `lz4`,`zstd` and `none`. `none` compression
algorithm will create an uncompressed archive.

``` ini
backup_compression = gzip|lz4|zstd|none
```

Barman requires the CLI utility for the selected compression algorithm
to be available on both the Barman server _and_ the PostgreSQL server.
The CLI utility is used to extract the backup label from the compressed
backup and to decompress the backup on the PostgreSQL server during
recovery. These can be installed through system packages named `gzip`,
`lz4` and `zstd` on Debian, Ubuntu, RedHat, CentOS and SLES systems.

> **Note:** On Ubuntu 18.04 (bionic) the `lz4` utility is available in
> the `liblz4-tool` pacakge.

> **Note:** `zstd` version must be 1.4.4 or higher. The system packages
> for `zstd` on Debian 10 (buster), Ubuntu 18.04 (bionic) and SLES 12
> install an earlier version - `backup_compression = zstd` will not
> work with these packages.

> **Note:** `lz4` and `zstd` are only available with PostgreSQL version
> 15 or higher.

> **IMPORTANT:** If you are using `backup_compression` you must also
> set `recovery_staging_path` so that `barman recover` is able to
> recover the compressed backups. See the [Recovering compressed backups](#recovering-compressed-backups)
> section for more information.

#### Compression workers

This optional parameter allows compression using multiple threads to increase compression speed (default being 0).

```ini
backup_compression_workers = 2
```

> **Note:** This option is only available with `zstd` compression.

> **Note:** `zstd` version must be 1.5.0 or higher. Or 1.4.4 or higher compiled with multithreading option.

#### Compression level

The compression level can be specified using the
`backup_compression_level` option. This should be set to an integer
value supported by the compression algorithm specified in
`backup_compression`.
If not defined, compression algorithm default value will be used.

`none` compression only supports `backup_compression_level=0`.

> **Note:** `backup_compression_level` available and default values depends on the compression algorithm used. 
> Please check the compression algorithm documentation for more details.

> **Note:** On PostgreSQL version prior to 15, `gzip` support `backup_compression_level=0`.
> It results using default compression level

#### Compression location

When using Barman with PostgreSQL version 15 or higher it is possible
to specify for compression to happen on the server (i.e. PostgreSQL
will compress the backup) or on the client (i.e. pg_basebackup
will compress the backup). This can be achieved using the
`backup_compression_location` option:

> **IMPORTANT:** the `backup_compression_location` option is only
> available when running against PostgreSQL 15 or later.

``` ini
backup_compression_location = server|client
```

Using `backup_compression_location = server` should reduce the network
bandwidth required by the backup at the cost of moving the compression
work onto the PostgreSQL server.

When `backup_compression_location` is set to `server` then an
additional option, `backup_compression_format`, can be set to `plain`
in order to have pg_basebackup uncompress the data before writing it
to disk:

#### Compression format

``` ini
backup_compression_format = plain|tar
```

If `backup_compression_format` is unset or has the value `tar` then
the backup will be written to disk as compressed tarballs. A description
of both the `plain` and `tar` formats can be found in the [pg_basebackup
documentation][pg_basebackup-documentation].

> **IMPORTANT:** Barman uses external tools to manage compressed backups.
> Depending on the `backup_compression` and `backup_compression_format`
> You may need to install one or more tools on the Postgres server and
> the Barman server.
> The following table will help you choose according to your configuration.

| **backup_compression** | **backup_compression_format** | **Postgres server** | **Barman server** |
|:---------:|:---------------------:|:-------------------------:|:----------------------:|
| gzip | plain | **tar** | None |
| gzip |  tar  | **tar** | **tar** |
| lz4 | plain | **tar, lz4** | None |
| lz4 |  tar  | **tar, lz4** | **tar, lz4** |
| zstd | plain | **tar, zstd** | None |
| zstd |  tar  | **tar, zstd** | **tar, zstd** |
| none |  tar  | **tar** | **tar** |

### Concurrent backup

Normally, during backup operations, Barman uses PostgreSQL native
functions `pg_start_backup` and `pg_stop_backup` for _concurrent
backup_.[^ABOUT_CONCURRENT_BACKUP] This is the recommended way of
taking backups for PostgreSQL 9.6 and above (though note the
functions have been renamed to `pg_backup_start` and `pg_backup_stop`
in the PostgreSQL 15 beta).

[^ABOUT_CONCURRENT_BACKUP]:
  Concurrent backup is a technology that uses the _streaming replication
  protocol_ (for example, using a tool like `pg_basebackup`).

As well as being the recommended backup approach, concurrent backup
also allows the following architecture scenario with Barman: **backup from
a standby server**, using `rsync`.

By default, `backup_options` is set to `concurrent_backup`. If exclusive
backup is required for PostgreSQL servers older than version 15 then users
should set `backup_options` to `exclusive_backup`.

When `backup_options` is set to `concurrent_backup`, Barman activates
the _concurrent backup mode_ for a server and follows these two simple
rules:

- `ssh_command` must point to the destination Postgres server
- `conninfo` must point to a database on the destination Postgres
  database.

> **IMPORTANT:** In case of a concurrent backup, currently Barman
> cannot determine whether the closing WAL file of a full backup has
> actually been shipped - opposite of an exclusive backup
> where PostgreSQL itself makes sure that the WAL file is correctly
> archived. Be aware that the full backup cannot be considered
> consistent until that WAL file has been received and archived by
> Barman. Barman 2.5 introduces a new state, called `WAITING_FOR_WALS`,
> which is managed by the `check-backup` command (part of the
> ordinary maintenance job performed by the `cron` command).
> From Barman 2.10, you can use the `--wait` option with `barman backup`
> command.

### Concurrent backup of a standby

If backing up a standby then the following [configuration options][config-options]
should point to the standby server:

- `conninfo`
- `streaming_conninfo` (when using `backup_method = postgres` or `streaming_archiver = on`)
- `ssh_command` (when using `backup_method = rsync`)

The following config option should point to the primary server:

- `primary_conninfo`

Barman will use `primary_conninfo` to switch to a new WAL on the primary
so that the concurrent backup against the standby can complete without
having to wait for a WAL switch to occur naturally.

> **NOTE:** It is especially important that `primary_conninfo` is
> set if the standby is to be backed up when there is little or no write
> traffic on the primary.

As of Barman 3.8.0, If `primary_conninfo` is set, is possible to add for a server a
`primary_checkpoint_timeout` option, which is the maximum time (in seconds)
for Barman to wait for a new WAL file to be produced
before forcing the execution of a checkpoint on the primary.
The `primary_checkpoint_timeout` option should be set to an amount of seconds
greater of the value of the `archive_timeout` option set on the primary server.

If `primary_conninfo` is not set then the
backup will still run however it will wait at the stop backup stage
until the current WAL segment on the primary is newer than the latest
WAL required by the backup.

Barman currently requires that WAL files and backup data come from the
same PostgreSQL server. In the case that the standby is promoted to primary
the backups and WALs will continue to be valid however you may wish to update
the Barman configuration so that it uses the new standby for taking backups and
receiving WALs.

WALs can be obtained from the standby using either WAL streaming or WAL
archiving. To use WAL streaming follow the instructions in the
[WAL streaming](#wal-streaming) section.

To use WAL archiving from the standby follow the instructions in the [WAL archiving via archive_command](#wal-archiving-via-archive_command) section
*and additionally* set `archive_mode = always` in the PostgreSQL config on
the standby server.

> **NOTE:** With PostgreSQL 10 and earlier Barman cannot handle WAL streaming
> and WAL archiving being enabled at the same time on a standby. You must therefore
> disable WAL archiving if using WAL streaming and vice versa. This is because it is
> possible for WALs produced by PostgreSQL 10 and earlier to be logically equivalent
> but differ at the binary level, causing Barman to fail to detect that two WALs are
> identical.

### Immediate checkpoint

Before starting a backup, Barman requests a checkpoint, which
generates additional workload. Normally that checkpoint is throttled
according to the settings for workload control on the PostgreSQL
server, which means that the backup could be delayed.

This default behaviour can be changed through the `immediate_checkpoint`
configuration global/server option (set to `false` by default).

If `immediate_checkpoint` is set to `true`, PostgreSQL will not try to
limit the workload, and the checkpoint will happen at maximum speed,
starting the backup as soon as possible.

At any time, you can override the configuration option behaviour, by
issuing `barman backup` with any of these two options:

- `--immediate-checkpoint`, which forces an immediate checkpoint;
- `--no-immediate-checkpoint`, which forces to wait for the checkpoint
  to happen.

### Local backup

> **DISCLAIMER:** This feature is not recommended for production usage,
> as Barman and PostgreSQL reside on the same server and are part of
> the same single point of failure.
> Some EnterpriseDB customers have requested to add support for
> local backup to Barman to be used under specific circumstances
> and, most importantly, under the 24/7 production service delivered
> by the company. Using this feature currently requires installation
> from sources, or to customise the environment for the `postgres`
> user in terms of permissions as well as logging and cron configurations.

Under special circumstances, Barman can be installed on the same
server where the PostgreSQL instance resides, with backup data stored
on a separate volume from PGDATA and, where applicable, tablespaces.
Usually, these volumes reside on network storage appliances, with
filesystems like NFS.

This architecture is not endorsed by EnterpriseDB.
For an enhanced business continuity experience of PostgreSQL, with better
results in terms of RPO and RTO, EnterpriseDB still recommends the
shared nothing architecture with a remote installation of Barman, capable
of acting like a witness server for replication and monitoring purposes.

The only requirement for local backup is that Barman runs with the
same user as the PostgreSQL server, which is normally `postgres`.
Given that the Community packages by default install Barman under the `barman`
user, this use case requires manual installation procedures that include:

- cron configurations
- log configurations, including logrotate

In order to use local backup for a given server in Barman, you need to
set `backup_method` to `local-rsync`. The feature is essentially identical
to its `rsync` equivalent, which relies on SSH instead and operates remotely.
With `local-rsync` file system copy is performed issuing `rsync` commands locally
(for this reason it is required that Barman runs with the same user as PostgreSQL).

An excerpt of configuration for local backup for a server named `local-pg13` is:

```ini
[local-pg13]
description = "Local PostgreSQL 13"
backup_method = local-rsync
...
```

## Archiving features

### WAL compression

The `barman cron` command will compress WAL files if the `compression`
option is set in the configuration file. This option allows five
values:

- `bzip2`: for Bzip2 compression (requires the `bzip2` utility)
- `gzip`: for Gzip compression (requires the `gzip` utility)
- `pybzip2`: for Bzip2 compression (uses Python's internal compression module)
- `pygzip`: for Gzip compression (uses Python's internal compression module)
- `pigz`: for Pigz compression (requires the `pigz` utility)
- `custom`: for custom compression, which requires you to set the
  following options as well:
      - `custom_compression_filter`: a compression filter
      - `custom_decompression_filter`: a decompression filter
      - `custom_compression_magic`: a hex string to identify a custom compressed wal file

> _NOTE:_ All methods but `pybzip2` and `pygzip` require `barman
> archive-wal` to fork a new process.

### Synchronous WAL streaming

Barman can also reduce the Recovery Point Objective to zero, by
collecting the transaction WAL files like a synchronous standby server
would.

To configure such a scenario, the Barman server must be configured to
archive WALs via the [streaming connection](#postgresql-streaming-connection),
and the `receive-wal` process should figure as a synchronous standby
of the PostgreSQL server.

First of all, you need to retrieve the application name of the Barman
`receive-wal` process with the `show-servers` command:

``` bash
barman@backup$ barman show-servers pg|grep streaming_archiver_name
 streaming_archiver_name: barman_receive_wal
```

Then the application name should be added to the `postgresql.conf`
file as a synchronous standby:

``` ini
synchronous_standby_names = 'barman_receive_wal'
```

> **IMPORTANT:** this is only an example of configuration, to show you that
> Barman is eligible to be a synchronous standby node.
> We are not suggesting to use ONLY Barman.
> You can read _["Synchronous Replication"][synch]_ from the PostgreSQL
> documentation for further information on this topic.

The PostgreSQL server needs to be restarted for the configuration to
be reloaded.

If the server has been configured correctly, the `replication-status`
command should show the `receive_wal` process as a synchronous
streaming client:

``` bash
[root@backup ~]# barman replication-status pg
Status of streaming clients for server 'pg':
  Current xlog location on master: 0/9000098
  Number of streaming clients: 1

  1. #1 Sync WAL streamer
     Application name: barman_receive_wal
     Sync stage      : 3/3 Remote write
     Communication   : TCP/IP
     IP Address      : 139.59.135.32 / Port: 58262 / Host: -
     User name       : streaming_barman
     Current state   : streaming (sync)
     Replication slot: barman
     WAL sender PID  : 2501
     Started at      : 2016-09-16 10:33:01.725883+00:00
     Sent location   : 0/9000098 (diff: 0 B)
     Write location  : 0/9000098 (diff: 0 B)
     Flush location  : 0/9000098 (diff: 0 B)
```

## Catalog management features

### Minimum redundancy safety

You can define the minimum number of periodic backups for a PostgreSQL
server, using the global/per server configuration option called
`minimum_redundancy`, by default set to 0.

By setting this value to any number greater than 0, Barman makes sure
that at any time you will have at least that number of backups in a
server catalog.

This will protect you from accidental `barman delete` operations.

> **IMPORTANT:**
> Make sure that your retention policy settings do not collide with
> minimum redundancy requirements. Regularly check Barman's log for
> messages on this topic.

### Retention policies

Barman supports **retention policies** for backups.

A backup retention policy is a user-defined policy that determines how
long backups and related archive logs (Write Ahead Log segments) need
to be retained for recovery procedures.

Based on the user's request, Barman retains the periodic backups
required to satisfy the current retention policy and any archived WAL
files required for the complete recovery of those backups.

Barman users can define a retention policy in terms of **backup
redundancy** (how many periodic backups) or a **recovery window** (how
long).

Retention policy based on redundancy

  : In a redundancy based retention policy, the user determines how
    many periodic backups to keep. A redundancy-based retention policy
    is contrasted with retention policies that use a recovery window.

Retention policy based on recovery window

  : A recovery window is one type of Barman backup retention policy,
    in which the DBA specifies a period of time and Barman ensures
    retention of backups and/or archived WAL files required for
    point-in-time recovery to any time during the recovery window. The
    interval always ends with the current time and extends back in
    time for the number of days specified by the user. For example, if
    the retention policy is set for a recovery window of seven days,
    and the current time is 9:30 AM on Friday, Barman retains the
    backups required to allow point-in-time recovery back to 9:30 AM
    on the previous Friday.

#### Scope

Retention policies can be defined for:

- **PostgreSQL periodic base backups**: through the `retention_policy`
  configuration option
- **Archive logs**, for Point-In-Time-Recovery: through the
  `wal_retention_policy` configuration option

> **IMPORTANT:**
> In a temporal dimension, archive logs must be included in the time
> window of periodic backups.

There are two typical use cases here: full or partial point-in-time
recovery.

Full point in time recovery scenario:

  : Base backups and archive logs share the same retention policy,
    allowing you to recover at any point in time from the first
    available backup.

Partial point in time recovery scenario:

  : Base backup retention policy is wider than that of archive logs,
    for example allowing users to keep full, weekly backups of the
    last 6 months, but archive logs for the last 4 weeks (granting to
    recover at any point in time starting from the last 4 periodic
    weekly backups).

> **IMPORTANT:**
> Currently, Barman implements only the **full point in time
> recovery** scenario, by constraining the `wal_retention_policy`
> option to `main`.

#### How they work

Retention policies in Barman can be:

- **automated**: enforced by `barman cron`
- **manual**: Barman simply reports obsolete backups and allows you
  to delete them

> **IMPORTANT:**
> Currently Barman does not implement manual enforcement. This feature
> will be available in future versions.

#### Configuration and syntax

Retention policies can be defined through the following configuration
options:

- `retention_policy`: for base backup retention
- `wal_retention_policy`: for archive logs retention
- `retention_policy_mode`: can only be set to `auto` (retention
  policies are automatically enforced by the `barman cron` command)

These configuration options can be defined both at a global level and
a server level, allowing users maximum flexibility on a multi-server
environment.

##### Syntax for `retention_policy`

The general syntax for a base backup retention policy through
`retention_policy` is the following:

``` ini
retention_policy = {REDUNDANCY value | RECOVERY WINDOW OF value {DAYS | WEEKS | MONTHS}}
```

Where:

- syntax is case insensitive
- `value` is an integer and is > 0
- in case of **redundancy retention policy**:
      - `value` must be greater than or equal to the server minimum
        redundancy level (if that value is not assigned,
        a warning is generated)
      - the first valid backup is the value-th backup in a reverse
        ordered time series
- in case of **recovery window policy**:
      - the point of recoverability is: current time - window
      - the first valid backup is the first available backup before
        the point of recoverability; its value in a reverse ordered
        time series must be greater than or equal to the server
        minimum redundancy level (if it is not assigned to that value
        and a warning is generated)

By default, `retention_policy` is empty (no retention enforced).

##### Syntax for `wal_retention_policy`

Currently, the only allowed value for `wal_retention_policy` is the
special value `main`, that maps the retention policy of archive logs
to that of base backups.

## Hook scripts

Barman allows a database administrator to run hook scripts on these
two events:

- before and after a backup
- before and after the deletion of a backup
- before and after a WAL file is archived
- before and after a WAL file is deleted

There are two types of hook scripts that Barman can manage:

- standard hook scripts
- retry hook scripts

The only difference between these two types of hook scripts is that
Barman executes a standard hook script only once, without checking its
return code, whereas a retry hook script may be executed more than
once, depending on its return code.

Specifically, when executing a retry hook script, Barman checks the
return code and retries indefinitely until the script returns either
`SUCCESS` (with standard return code `0`), or `ABORT_CONTINUE` (return
code `62`), or `ABORT_STOP` (return code `63`). Barman treats any
other return code as a transient failure to be retried. Users are
given more power: a hook script can control its workflow by specifying
whether a failure is transient. Also, in case of a 'pre' hook script,
by returning `ABORT_STOP`, users can request Barman to interrupt the
main operation with a failure.

Hook scripts are executed in the following order:

1. The standard 'pre' hook script (if present)
2. The retry 'pre' hook script (if present)
3. The actual event (i.e. backup operation, or WAL archiving), if
   retry 'pre' hook script was not aborted with `ABORT_STOP`
4. The retry 'post' hook script (if present)
5. The standard 'post' hook script (if present)

The output generated by any hook script is written in the log file of
Barman.

> **NOTE:**
> Currently, `ABORT_STOP` is ignored by retry 'post' hook scripts. In
> these cases, apart from logging an additional warning, `ABORT_STOP`
> will behave like `ABORT_CONTINUE`.

### Backup scripts

These scripts can be configured with the following global
configuration options (which can be overridden on a per server basis):

- `pre_backup_script`: _hook script_ executed _before_ a base backup,
  only once, with no check on the exit code
- `pre_backup_retry_script`: _retry hook script_ executed _before_ a
  base backup, repeatedly until success or abort
- `post_backup_retry_script`: _retry hook script_ executed _after_ a
  base backup, repeatedly until success or abort
- `post_backup_script`: _hook script_ executed _after_ a base backup,
  only once, with no check on the exit code

The script definition is passed to a shell and can return any exit
code. Only in case of a _retry_ script, Barman checks the return code
(see the [hook script section](#hook_scripts)).

The shell environment will contain the following variables:

- `BARMAN_BACKUP_DIR`: backup destination directory
- `BARMAN_BACKUP_ID`: ID of the backup
- `BARMAN_CONFIGURATION`: configuration file used by Barman
- `BARMAN_ERROR`: error message, if any (only for the `post` phase)
- `BARMAN_PHASE`: phase of the script, either `pre` or `post`
- `BARMAN_PREVIOUS_ID`: ID of the previous backup (if present)
- `BARMAN_RETRY`: `1` if it is a retry script, `0` if not
- `BARMAN_SERVER`: name of the server
- `BARMAN_STATUS`: status of the backup
- `BARMAN_VERSION`: version of Barman

### Backup delete scripts

Version **2.4** introduces pre and post backup delete scripts.

As previous scripts, backup delete scripts can be configured within global
configuration options, and it is possible to override them on a per server
basis:

- `pre_delete_script`: _hook script_ launched _before_ the deletion
  of a backup, only once, with no check on the exit code
- `pre_delete_retry_script`: _retry hook script_ executed _before_
  the deletion of a backup, repeatedly until success or abort
- `post_delete_retry_script`: _retry hook script_ executed _after_
  the deletion of a backup, repeatedly until success or abort
- `post_delete_script`: _hook script_ launched _after_ the deletion
  of a backup, only once, with no check on the exit code

The script is executed through a shell and can return any exit code.
Only in case of a _retry_ script, Barman checks the return code (see
the upper section).

Delete scripts uses the same environmental variables of a backup script,
plus:

- `BARMAN_NEXT_ID`: ID of the next backup (if present)

### WAL archive scripts

Similar to backup scripts, archive scripts can be configured with
global configuration options (which can be overridden on a per server
basis):

- `pre_archive_script`: _hook script_ executed _before_ a WAL file is
  archived by maintenance (usually `barman cron`), only once, with no
  check on the exit code
- `pre_archive_retry_script`: _retry hook script_ executed _before_ a
  WAL file is archived by maintenance (usually `barman cron`),
  repeatedly until it is successful or aborted
- `post_archive_retry_script`: _retry hook script_ executed _after_ a
  WAL file is archived by maintenance, repeatedly until it is
  successful or aborted
- `post_archive_script`: _hook script_ executed _after_ a WAL file is
  archived by maintenance, only once, with no check on the exit code

The script is executed through a shell and can return any exit code.
Only in case of a _retry_ script, Barman checks the return code (see
the upper section).

Archive scripts share with backup scripts some environmental
variables:

- `BARMAN_CONFIGURATION`: configuration file used by Barman
- `BARMAN_ERROR`: error message, if any (only for the `post` phase)
- `BARMAN_PHASE`: phase of the script, either `pre` or `post`
- `BARMAN_SERVER`: name of the server

Following variables are specific to archive scripts:

- `BARMAN_SEGMENT`: name of the WAL file
- `BARMAN_FILE`: full path of the WAL file
- `BARMAN_SIZE`: size of the WAL file
- `BARMAN_TIMESTAMP`: WAL file timestamp
- `BARMAN_COMPRESSION`: type of compression used for the WAL file

### WAL delete scripts

Version **2.4** introduces pre and post WAL delete scripts.

Similarly to the other hook scripts, wal delete scripts can be
configured with global configuration options, and is possible to
override them on a per server basis:

- `pre_wal_delete_script`: _hook script_ executed _before_
  the deletion of a WAL file
- `pre_wal_delete_retry_script`: _retry hook script_ executed _before_
  the deletion of a WAL file, repeatedly until it is successful
  or aborted
- `post_wal_delete_retry_script`: _retry hook script_ executed _after_
  the deletion of a WAL file, repeatedly until it is successful
  or aborted
- `post_wal_delete_script`: _hook script_ executed _after_
  the deletion of a WAL file

The script is executed through a shell and can return any exit code.
Only in case of a _retry_ script, Barman checks the return code (see
the upper section).

WAL delete scripts use the same environmental variables as WAL archive
scripts.

### Recovery scripts

Version **2.4** introduces pre and post recovery scripts.

As previous scripts, recovery scripts can be configured within global
configuration options, and is possible to override them on a per server
basis:

- `pre_recovery_script`: _hook script_ launched _before_ the recovery
  of a backup, only once, with no check on the exit code
- `pre_recovery_retry_script`: _retry hook script_ executed _before_
  the recovery of a backup, repeatedly until success or abort
- `post_recovery_retry_script`: _retry hook script_ executed _after_
  the recovery of a backup, repeatedly until success or abort
- `post_recovery_script`: _hook script_ launched _after_ the recovery
  of a backup, only once, with no check on the exit code

The script is executed through a shell and can return any exit code.
Only in case of a _retry_ script, Barman checks the return code (see
the upper section).

Recovery scripts uses the same environmental variables of a backup
script, plus:

- `BARMAN_DESTINATION_DIRECTORY`: the directory where the new instance
  is recovered

- `BARMAN_TABLESPACES`: tablespace relocation map (JSON, if present)

- `BARMAN_REMOTE_COMMAND`: secure shell command used
  by the recovery (if present)

- `BARMAN_RECOVER_OPTIONS`: recovery additional options (JSON, if present)

## Customization

### Lock file directory

Barman allows you to specify a directory for lock files through the
`barman_lock_directory` global option.

Lock files are used to coordinate concurrent work at global and server
level (for example, cron operations, backup operations, access to the
WAL archive, and so on.).

By default (for backward compatibility reasons),
`barman_lock_directory` is set to `barman_home`.

> **TIP:**
> Users are encouraged to use a directory in a volatile partition,
> such as the one dedicated to run-time variable data (e.g.
> `/var/run/barman`).

### Binary paths

As of version 1.6.0, Barman allows users to specify one or more directories
where Barman looks for executable files, using the global/server
option `path_prefix`.

If a `path_prefix` is provided, it must contain a list of one or more
directories separated by colon. Barman will search inside these directories
first, then in those specified by the `PATH` environment variable.

By default the `path_prefix` option is empty.

## Integration with cluster management systems

Barman has been designed for integration with standby servers (with
streaming replication or traditional file based log shipping) and high
availability tools like [repmgr][repmgr].

From an architectural point of view, PostgreSQL must be configured to
archive WAL files directly to the Barman server.
Barman, thanks to the `get-wal` framework, can also be used as a WAL hub.
For this purpose, you can use the `barman-wal-restore` script, part
of the `barman-cli` package, with all your standby servers.

The `replication-status` command allows
you to get information about any streaming client attached to the
managed server, in particular hot standby servers and WAL streamers.

### Configuration Models

Configuration models define a set of overrides for configuration options. These
overrides can be applied to Barman servers which are part of the same cluster as
the config models. They can be useful when handling clustered environments, so you
can change the configuration of a Barman server in response to failover events,
for example.

As an example, let's say you have a PostgreSQL cluster with the following nodes:

* `pg-node-1`: primary
* `pg-node-2`: standby
* `pg-node-3`: standby

Assume you are backing up from the primary node, and have a configuration which
includes the following options:

```ini
[my-barman-server]
cluster = my-cluster
conninfo = host=pg-node-1 user=barman database=postgres
streaming_conninfo = host=pg-node-1 user=streaming_barman
; other options...
```

You could, for example, have a configuration model for that cluster as follows:

```ini
[my-barman-server:backup-from-pg-node-2]
cluster = my-cluster
model = true
conninfo = host=pg-node-2 user=barman database=postgres
streaming_conninfo = host=pg-node-2 user=streaming_barman
```

Which could be applied upon a failover from `pg-node-1` to `pg-node-2` with
the following command, so you start backing up from the new primary node:

```bash
barman config-switch my-barman-server my-barman-server:backup-from-pg-node-2
```

That will override the cluster configuration options with the values
defined in the selected model.

> *NOTE*: not all options are configurable through models. Please refer to
> [section 5 of the 'man' page][man5] to check settings which scope applies to
> models.

> *NOTE*: you might be interested in checking [pg-backup-api](https://www.enterprisedb.com/docs/supported-open-source/barman/pg-backup-api/),
> which can start a REST API and listen for remote requests for executing
> `barman` commands, including `barman config-switch`.

## Parallel jobs

By default, Barman uses only one worker for file copy during both backup and
recover operations. Starting from version 2.2, it is possible to customize the
number of workers that will perform file copy. In this case, the
files to be copied will be equally distributed among all parallel workers.

It can be configured in global and server scopes, adding these in the
corresponding configuration file:

``` ini
parallel_jobs = n
```

where `n` is the desired number of parallel workers to be used in file copy
operations. The default value is 1.

In any case, users can override this value at run-time when executing
`backup` or `recover` commands. For example, you can use 4 parallel workers
as follows:

``` bash
barman backup --jobs 4 server1
```

Or, alternatively:

``` bash
barman backup --j 4 server1
```

Please note that this parallel jobs feature is only available for servers
configured through `rsync`/SSH. For servers configured through streaming
protocol, Barman will rely on `pg_basebackup` which is currently limited
to only one worker.

### Parallel jobs and sshd MaxStartups

Barman limits the rate at which parallel Rsync jobs are started in order to
avoid exceeding the maximum number of concurrent unauthenticated connections
allowed by the SSH server. This maximum is defined by the sshd parameter
`MaxStartups` - if more than `MaxStartups` connections have been created but
not yet authenticated then the SSH server may drop some or all of the
connections resulting in a failed backup or recovery.

The default value of sshd `MaxStartups` on most platforms is 10. Barman
therefore starts parallel jobs in batches of 10 and does not start more than
one batch of jobs within a one second time period. This yields an effective
rate limit of 10 jobs per second.

This limit can be changed using the following two configuration options:

- `parallel_jobs_start_batch_size`: The maximum number of parallel jobs to
  start in a single batch.
- `parallel_jobs_start_batch_period`: The time period in seconds over which a
  single batch of jobs will be started.

For example, to ensure no more than five new Rsync jobs will be created within
any two second time period:

``` ini
parallel_jobs_start_batch_size = 5
parallel_jobs_start_batch_period = 2
```

The configuration options can be overridden using the following arguments
with both `barman backup` and `barman recover` commands:

- `--jobs-start-batch-size`
- `--jobs-start-batch-period`

## Geographical redundancy

It is possible to set up **cascading backup architectures** with Barman,
where the source of a backup server
is a Barman installation rather than a PostgreSQL server.

This feature allows users to transparently keep _geographically distributed_
copies of PostgreSQL backups.

In Barman jargon, a backup server that is connected to a Barman installation
rather than a PostgreSQL server is defined **passive node**.
A passive node is configured through the `primary_ssh_command` option, available
both at global (for a full replica of a primary Barman installation) and server
level (for mixed scenarios, having both _direct_ and _passive_ servers).

### Sync information

The `barman sync-info` command is used to collect information regarding the
current status of a Barman server that is useful for synchronisation purposes.
The available syntax is the following:

``` bash
barman sync-info [--primary] <server_name> [<last_wal> [<last_position>]]
```

The command returns a JSON object containing:

- A map with all the backups having status `DONE` for that server
- A list with all the archived WAL files
- The configuration for the server
- The last read position (in the _xlog database file_)
- the name of the last read WAL file

The JSON response contains all the required information for the synchronisation
between the `master` and a `passive` node.

If `--primary` is specified, the command is executed on the defined
primary node, rather than locally.

### Configuration

Configuring a server as `passive node` is a quick operation.
Simply add to the server configuration the following option:

``` ini
primary_ssh_command = ssh barman@primary_barman
```

This option specifies the SSH connection parameters to the primary server,
identifying the source of the backup data for the passive server.

If you are invoking barman with the `-c/--config` option and you want to use
the same option when the passive node invokes barman on the primary node then
add the following option:

``` ini
forward_config_path = true
```

### Node synchronisation

When a node is marked as `passive` it is treated in a special way by Barman:

- it is excluded from standard maintenance operations
- direct operations to PostgreSQL are forbidden, including `barman backup`

Synchronisation between a passive server and its primary is automatically
managed by `barman cron` which will transparently invoke:

1. `barman sync-info --primary`, in order to collect synchronisation information
2. `barman sync-backup`, in order to create a local copy of every backup that is available on the primary node
3. `barman sync-wals`, in order to copy locally all the WAL files available on the primary node

### Manual synchronisation

Although `barman cron` automatically manages passive/primary node
synchronisation, it is possible to manually trigger synchronisation
of a backup through:

``` bash
barman sync-backup <server_name> <backup_id>
```

Launching `sync-backup` barman will use the primary_ssh_command to connect to the master server, then
if the backup is present on the remote machine, will begin to copy all the files using rsync.
Only one single synchronisation process per backup is allowed.

WAL files also can be synchronised, through:

``` bash
barman sync-wals <server_name>
```

## Cloud snapshot backups

Snapshot backups are backups which consist of one or more snapshots of cloud storage volumes.

A snapshot backup can be taken for a [suitable PostgreSQL server](#prerequisites-for-cloud-snapshots) using either of the following commands:

- `barman backup` with the [required configuration operations for snapshots](#configuration-for-snapshot-backups) if a Barman server is being used to store WALs and backup metadata.
- `barman-cloud-backup` with the required [command line arguments](#barman-cloud-backup-for-snapshots) if there is no Barman server and instead a cloud object store is being used for WALs and backup metadata.

### Snapshot backup details

The high level process for taking a snapshot backup is as follows:

1. Barman carries out a series of pre-flight checks to validate the snapshot options, instance and disks.
2. Barman starts a backup using the [PostgreSQL backup API][postgres-low-level-base-backup].
3. The cloud provider API is used to trigger a snapshot for each specified disk. Barman will wait until the snapshot has reached the required state for guaranteeing application consistency before moving on to the next disk.
4. Additional provider-specific data, such as the device name for each disk, is saved to the backup metadata.
5. The mount point and mount options for each disk are saved in the backup metadata.
6. Barman stops the backup using the PostgreSQL backup API.

The cloud provider API calls are made on the node where the backup command runs; this will be either the Barman server (when `barman backup` is used) or the PostgreSQL server (when `barman-cloud-backup` is used).

The following pre-flight checks are carried out before each backup and also when `barman check` runs against a server configured for snapshot backups:

- The compute instance specified by `snapshot_instance` and any provider-specific arguments exists.
- The disks specified by `snapshot_disks` exist.
- The disks specified by `snapshot_disks` are attached to `snapshot_instance`.
- The disks specified by `snapshot_disks` are mounted on `snapshot_instance`.

### Recovering from a snapshot backup

Barman will not currently perform a fully automated recovery from snapshot backups.
This is because recovery from snapshots requires the provision and management of new infrastructure which is something better handled by dedicated infrastructure-as-code solutions such as Terraform.

However, the `barman recover` command can still be used to validate the snapshot recovery instance, carry out post-recovery tasks such as checking the PostgreSQL configuration for unsafe options and set any required PITR options.
It will also copy the backup_label file into place (since the backup label is not stored in any of the volume snapshots) and copy across any required WALs (unless the `--get-wal` recovery option is used, in which case it will configure the PostgreSQL `restore_command` to fetch the WALs).

If restoring a backup made with `barman-cloud-backup` then the more limited [barman-cloud-restore](#barman-cloud-restore-for-snapshots) command should be used instead of `barman recover`.

Recovery from a snapshot backup consists of the following steps:

1. Provision a new disk for each snapshot taken during the backup.
2. Provision a compute instance where each disk provisioned in step 1 is attached and mounted according to the backup metadata.
3. Use the [barman recover](#recover) or [barman-cloud-restore](#barman-cloud-restore-for-snapshots) command to validate and finalize the recovery.

Steps 1 and 2 are best handled by an existing infrastructure-as-code system however it is also possible to carry these steps out manually or using a custom script.

The following resources may be helpful when carrying out these steps:

- An example [recovery script for GCP][snapshot-recovery-script].
- An example [runbook for Azure][snapshot-recovery-runbook-azure].

The above resources make assumptions about the backup/recovery environment and should not be considered suitable for production use without further customization.

Once the recovery instance is provisioned and disks cloned from the backup snapshots are attached and mounted, run `barman recover` with the following additional arguments:

- `--remote-ssh-command`: The ssh command required to log in to the recovery instance.
- `--snapshot-recovery-instance`: The name of the recovery instance as required by the cloud provider.
- Any additional arguments specific to the snapshot provider.

For example:

``` bash
barman recover SERVER_NAME BACKUP_ID REMOTE_RECOVERY_DIRECTORY \
    --remote-ssh-command 'ssh USER@HOST' \
    --snapshot-recovery-instance INSTANCE_NAME
```

Barman will automatically detect that the backup is a snapshot backup and check that the attached disks were cloned from the snapshots for that backup.
Barman will then prepare PostgreSQL for recovery by copying the backup label and WALs into place and setting any required recovery options in the PostgreSQL configuration.

The following additional `barman recover` arguments are available with the `gcp` provider:

- `--gcp-zone`: The name of the availability zone in which the recovery instance is located. If not provided then Barman will use the value of `gcp_zone` set in the server config.

The following additional `barman recover` arguments are available with the `azure` provider:

- `--azure-resource-group`: The resource group to which the recovery instance belongs. If not provided then Barman will use the value of `azure_resource_group` set in the server config.

The following additional `barman recover` arguments are available with the `aws` provider:

- `--aws-region`: The AWS region in which the recovery instance is located. If not provided then Barman will use the value of `aws_region` set in the server config.

Note the following `barman recover` arguments / config variables are unavailable when recovering snapshot backups:

| **Command argument**      | **Config variable** .   |
|:-------------------------:|:-----------------------:|
| `--bwlimit`               | `bandwidth_limit`       |
| `--jobs`                  | `parallel_jobs`         |
| `--recovery-staging-path` | `recovery_staging_path` |
| `--tablespace`            | N/A                     |

### Backup metadata for snapshot backups

Whether the recovery disks and instance are provisioned via infrastructure-as-code, ad-hoc automation or manually, it will be necessary to query Barman to find the snapshots required for a given backup.
This can be achieved using [barman show-backup](#show-backup) which will provide details for each snapshot in the backup.
For example:

``` bash
$ barman show-backup primary 20230123T131430
Backup 20230123T131430:
  Server Name            : primary
  System Id              : 7190784995399903779
  Status                 : DONE
  PostgreSQL Version     : 140006
  PGDATA directory       : /opt/postgres/data

  Snapshot information:
    provider             : gcp
    project              : project_id

    device_name          : pgdata
    snapshot_name        : barman-av-ubuntu20-primary-pgdata-20230123t131430
    snapshot_project     : project_id
    Mount point          : /opt/postgres
    Mount options        : rw,noatime

    device_name          : tbs1
    snapshot_name        : barman-av-ubuntu20-primary-tbs1-20230123t131430
    snapshot_project     : project_id
    Mount point          : /opt/postgres/tablespaces/tbs1
    Mount options        : rw,noatime
```

The the `--format=json` option can be used when integrating with external tooling, e.g.:

``` bash
$ barman --format=json show-backup primary 20230123T131430
...
"snapshots_info": {
  "provider": "gcp",
  "provider_info": {
    "project": "project_id"
  },
  "snapshots": [
    {
      "mount": {
        "mount_options": "rw,noatime",
        "mount_point": "/opt/postgres"
      },
      "provider": {
        "device_name": "pgdata",
        "snapshot_name": "barman-av-ubuntu20-primary-pgdata-20230123t131430",
        "snapshot_project": "project_id"
      }
    },
    {
      "mount": {
        "mount_options": "rw,noatime",
        "mount_point": "/opt/postgres/tablespaces/tbs1"
      },
      "provider": {
        "device_name": "tbs1",
        "snapshot_name": "barman-av-ubuntu20-primary-tbs1-20230123t131430",
        "snapshot_project": "project_id",
      }
    }
  ]
}
...
```

For backups taken with `barman-cloud-backup` there is an analogous [barman-cloud-backup-show][pgbarman-barman-cloud-backup-show] command which can be used along with `barman-cloud-backup-list` to query the backup metadata in the cloud object store.

The metadata available in `snapshots_info/provider_info` and `snapshots_info/snapshots/*/provider` varies by cloud provider as explained in the following sections.

#### GCP provider-specific metadata

The following fields are available in `snapshots_info/provider_info`:

- `project`: The GCP project ID of the project which owns the resources involved in backup and recovery.

The following fields are available in `snapshots_info/snapshots/*/provider`:

- `device_name`: The short device name with which the source disk for the snapshot was attached to the backup VM at the time of the backup.
- `snapshot_name`: The name of the snapshot.
- `snapshot_project`: The GCP project ID which owns the snapshot.

#### Azure provider-specific metadata

The following fields are available in `snapshots_info/provider_info`:

- `subscription_id`: The Azure subscription ID which owns the resources involved in backup and recovery.
- `resource_group`: The Azure resource group to which the resources involved in the backup belong.

The following fields are available in `snapshots_info/snapshots/*/provider`:

- `location`: The Azure location of the disk from which the snapshot was taken.
- `lun`: The LUN identifying the disk from which the snapshot was taken at the time of the backup.
- `snapshot_name`: The name of the snapshot.

#### AWS provider-specific metadata

The following fields are available in `snapshots_info/provider_info`:

- `account_id`: The ID of the AWS account which owns the resources used to make the backup.
- `region`: The AWS region in which the resources involved in backup are located.

The following fields are available in `snapshots_info/snapshots/*/provider`:

- `device_name`: The device to which the source disk was mapped on the backup VM at the time of the backup.
- `snapshot_id`: The ID of the snapshot as assigned by AWS.
- `snapshot_name`: The name of the snapshot.
