‘Multiple Repositories’

A key feature of the v2.33 pgBackrest release is support for multiple repositories. This lets you define different behavior for different backup repositories. As an example, you could have a local repository for fast restore that had short retention period (to save space) paired with a remote repository which is larger for cold storage.

This feature introduces a new –repo option which can be used to set the repository the command should apply to. All repo*- options are indexed to enable configuring multiple repositories.

In this section, we will set up a demo cluster (using EDBPostgresAdvancedServer version 13 on CentOS 7) to demonstrate how this new feature impacts each pgBackRest command.

Configuration

It must be noted that in this demo, we will set up two local repositories on our database host. For production purposes, it is recommended to store each repository on separate storage, and configure at least one remote repository.

[global]
#  repository details

repo1-path=/var/lib/edb/as13/backups/repo1
repo1-retention-full=1
repo2-path=/var/lib/edb/as13/backups/repo2
repo2-retention-full=1

#  general options

process-max=2
log-level-console=info
log-level-file=debug
start-fast=y
delta=y

[demo]
pg1-path=/var/lib/edb/as13/data
pg1-user=enterprisedb
pg1-port=5444

For safety reasons, the --repo option can’t be defined in the configuration file or a warning will be triggered: WARN: configuration file contains command-line only option 'repo' . If there is more than one repository configured and the --repo option is not specified for a command, the repository with the highest priority order (e.g. repo1 then repo2 ) will be chosen by default.

The --repo option is not required when only repo1 is configured to maintain backward compatibility. However, when a single repository is configured, it is recommended to use repo1 in the configuration.

Initialization

Stanza Create Command

The stanza-create command will automatically operate on all configured repositories:

$ pgbackrest --stanza=demo stanza-create
P00   INFO: stanza-create for stanza demo on repo1
P00   INFO: stanza-create for stanza demo on repo2

Check Command

The check command will trigger a new WAL segment to be archived and try to push it to all defined repositories:

$ pgbackrest --stanza=demo check
P00   INFO: check repo1 configuration (primary)
P00   INFO: check repo2 configuration (primary)
P00   INFO: check repo1 archive for WAL (primary)
P00   INFO: WAL segment ... successfully archived to ... on repo1
P00   INFO: check repo2 archive for WAL (primary)
P00   INFO: WAL segment ... successfully archived to ... on repo2

Archive Push Command

The archive-push command will always archive WALs in all configured repositories. Backups will need to be scheduled individually for each repository. In most cases, this is desirable since backup types and retention could vary per repository.

The archive_command can still be defined as usual:

$ psql -d postgres -c "show archive_command;"
             archive_command
- -----------------------------------------
 pgbackrest --stanza=demo archive-push %p
(1 row)

Here is a DEBUG extract of the PostgreSQL logs showing the archive-push activity:

...
P00  DEBUG:     storage/storage::storageNewWrite: => {
    type: posix, name: {"/var/lib/edb/as13/backups/repo1/archive/demo/13-1/0000000100000000/
                            000000010000000000000005-be9a61a800934879842fa647b0d50385f44e0228.gz"},
    modeFile: 0640, modePath: 0750, createPath: true, syncFile: true, syncPath: true, atomic: true}
...
P00  DEBUG:     storage/storage::storageNewWrite: => {
    type: posix, name: {"/var/lib/edb/as13/backups/repo2/archive/demo/13-1/0000000100000000/
                            000000010000000000000005-be9a61a800934879842fa647b0d50385f44e0228.gz"},
    modeFile: 0640, modePath: 0750, createPath: true, syncFile: true, syncPath: true, atomic: true}
...
P00   INFO: pushed WAL file 000000010000000000000005 to the archive

The archive-push command will try to push the WAL archive to all reachable repositories. The idea is to archive to as many repositories as possible even if we still need to throw an error to PostgreSQL to prevent it from removing the WAL file.

P00   INFO: archive-push command begin: [pg_wal/000000010000000000000006] ...
...
P00   INFO: archive-push command begin: [pg_wal/000000010000000000000007] ...
ERROR: [104]: archive-push command encountered error(s):
       repo2: [PathOpenError] unable to list file info for path ...: [13] Permission denied
P00   INFO: archive-push command end: aborted with exception [104]
DETAIL:  The failed archive command was: pgbackrest --stanza=demo archive-push pg_wal/000000010000000000000007
WARNING:  archiving write-ahead log file "000000010000000000000007" failed too many times, will try again later

The PostgreSQL archiver process should then report an error:

$ ps -o pid,cmd fx |grep archiver
 4440  \_ postgres: archiver failed on 000000010000000000000007

The next WAL segments should not then be archived:

$ psql -d postgres -c "SELECT pg_create_restore_point(generate some activity); SELECT pg_switch_wal();"
 pg_switch_wal
- --------------
 0/80001C8
(1 row)

$ ls as13/data/pg_wal/archive_status/ |grep ".ready"
000000010000000000000007.ready
000000010000000000000008.ready

This becomes very handy by adding archive-async=y to the configuration in order to use asynchronous archiving processes within pgBackRest itself. Even if PostgreSQL archiver process is still stuck, the archives will reach the working repositories:

$ $ ls /var/lib/edb/as13/backups/repo1/archive/demo/13-1/0000000100000000/
000000010000000000000005-be9a61a800934879842fa647b0d50385f44e0228.gz
000000010000000000000006-7108296955e1208c93447dcfc5712ce59af97907.gz
000000010000000000000007-f1b6fcdd23e96a19a7a89dc92613674c2316aa49.gz
000000010000000000000008-791c44d39ef772d58e51d64c8e3589231d3916a4.gz
000000010000000000000009-47241214eb3fd5e693eed778caa7b304a778ecc2.gz

$ ls /var/lib/edb/as13/backups/repo2/archive/demo/13-1/0000000100000000/
000000010000000000000005-be9a61a800934879842fa647b0d50385f44e0228.gz
000000010000000000000006-7108296955e1208c93447dcfc5712ce59af97907.gz

Let us unblock the archiver process (and remove asynchronous archiving) before going further:

$ $ ps -o pid,cmd fx |grep archiver
 4440  \_ postgres: archiver last was 00000001000000000000000

Backups

Backup Command

Let us take a few backups:

$ pgbackrest backup --stanza=demo --type=full
P00   INFO: backup command begin ...
P00   INFO: repo option not specified, defaulting to repo1
P00   INFO: execute non-exclusive pg_start_backup(): backup begins after the requested immediate checkpoint completes
P00   INFO: backup start archive = 00000001000000000000000C, lsn = 0/C000060
P00   INFO: full backup size = 50.4MB
P00   INFO: execute non-exclusive pg_stop_backup() and wait for all WAL segments to archive
P00   INFO: backup stop archive = 00000001000000000000000C, lsn = 0/C000138
P00   INFO: check archive for segment(s) 00000001000000000000000C:00000001000000000000000C
P00   INFO: new backup label = 20210419-142212F
P00   INFO: backup command end: completed successfully

$ pgbackrest backup --stanza=demo --type=full --repo=2
P00   INFO: backup command begin ...
P00   INFO: execute non-exclusive pg_start_backup(): backup begins after the requested immediate checkpoint completes
P00   INFO: backup start archive = 00000001000000000000000E, lsn = 0/E000028
P00   INFO: full backup size = 50.5MB
P00   INFO: execute non-exclusive pg_stop_backup() and wait for all WAL segments to archive
P00   INFO: backup stop archive = 00000001000000000000000E, lsn = 0/E000138
P00   INFO: check archive for segment(s) 00000001000000000000000E:00000001000000000000000E
P00   INFO: new backup label = 20210419-142414F
P00   INFO: backup command end: completed successfully

$ pgbackrest backup --stanza=demo --type=incr --repo=1
P00   INFO: backup command begin ...
P00   INFO: last backup label = 20210419-142212F, version = ...
P00   INFO: execute non-exclusive pg_start_backup(): backup begins after the requested immediate checkpoint completes
P00   INFO: backup start archive = 000000010000000000000010, lsn = 0/10000028
P00   INFO: incr backup size = 50.5MB
P00   INFO: execute non-exclusive pg_stop_backup() and wait for all WAL segments to archive
P00   INFO: backup stop archive = 000000010000000000000010, lsn = 0/10000100
P00   INFO: check archive for segment(s) 000000010000000000000010:000000010000000000000010
P00   INFO: new backup label = 20210419-142212F_20210419-142502I
P00   INFO: backup command end: completed successfully

$ pgbackrest backup --stanza=demo --type=incr --repo=2
P00   INFO: backup command begin ...
P00   INFO: last backup label = 20210419-142414F, version = ...
P00   INFO: execute non-exclusive pg_start_backup(): backup begins after the requested immediate checkpoint completes
P00   INFO: backup start archive = 000000010000000000000012, lsn = 0/12000028
P00   INFO: incr backup size = 50.6MB
P00   INFO: execute non-exclusive pg_stop_backup() and wait for all WAL segments to archive
P00   INFO: backup stop archive = 000000010000000000000012, lsn = 0/12000100
P00   INFO: check archive for segment(s) 000000010000000000000012:000000010000000000000012
P00   INFO: new backup label = 20210419-142414F_20210419-142556I
P00   INFO: backup command end: completed successfully

Here, we alternated full and incr backups in each repository.

Info Command

$ pgbackrest info --stanza=demo
stanza: demo
    status: ok
    cipher: none

    db (current)
        wal archive min/max (13): 000000010000000000000002/000000010000000000000012

        full backup: 20210419-142212F
            timestamp start/stop: 2021-04-19 14:22:12 / 2021-04-19 14:22:26
            wal start/stop: 00000001000000000000000C / 00000001000000000000000C
            database size: 50.4MB, database backup size: 50.4MB
            repo1: backup set size: 8MB, backup size: 8MB

        full backup: 20210419-142414F
            timestamp start/stop: 2021-04-19 14:24:14 / 2021-04-19 14:24:26
            wal start/stop: 00000001000000000000000E / 00000001000000000000000E
            database size: 50.5MB, database backup size: 50.5MB
            repo2: backup set size: 8MB, backup size: 8MB

        incr backup: 20210419-142212F_20210419-142502I
            timestamp start/stop: 2021-04-19 14:25:02 / 2021-04-19 14:25:04
            wal start/stop: 000000010000000000000010 / 000000010000000000000010
            database size: 50.5MB, database backup size: 613.4KB
            repo1: backup set size: 8MB, backup size: 26.3KB
            backup reference list: 20210419-142212F

        incr backup: 20210419-142414F_20210419-142556I
            timestamp start/stop: 2021-04-19 14:25:56 / 2021-04-19 14:25:57
            wal start/stop: 000000010000000000000012 / 000000010000000000000012
            database size: 50.6MB, database backup size: 661.5KB
            repo2: backup set size: 8MB, backup size: 28KB
            backup reference list: 20210419-142414F

The default order will sort backups by dates mixing the repositories. It might be confusing to find the backups depending on each other.

We can then split this view per repository:

$ pgbackrest info --stanza=demo --repo=1
stanza: demo
    status: ok
    cipher: none

    db (current)
        wal archive min/max (13): 00000001000000000000000C/000000010000000000000012

        full backup: 20210419-142212F
            timestamp start/stop: 2021-04-19 14:22:12 / 2021-04-19 14:22:26
            wal start/stop: 00000001000000000000000C / 00000001000000000000000C
            database size: 50.4MB, database backup size: 50.4MB
            repo1: backup set size: 8MB, backup size: 8MB

        incr backup: 20210419-142212F_20210419-142502I
            timestamp start/stop: 2021-04-19 14:25:02 / 2021-04-19 14:25:04
            wal start/stop: 000000010000000000000010 / 000000010000000000000010
            database size: 50.5MB, database backup size: 613.4KB
            repo1: backup set size: 8MB, backup size: 26.3KB
            backup reference list: 20210419-142212F

$ pgbackrest info --stanza=demo --repo=2
stanza: demo
    status: ok
    cipher: none

    db (current)
        wal archive min/max (13): 00000001000000000000000E/000000010000000000000012

        full backup: 20210419-142414F
            timestamp start/stop: 2021-04-19 14:24:14 / 2021-04-19 14:24:26
            wal start/stop: 00000001000000000000000E / 00000001000000000000000E
            database size: 50.5MB, database backup size: 50.5MB
            repo2: backup set size: 8MB, backup size: 8MB

        incr backup: 20210419-142414F_20210419-142556I
            timestamp start/stop: 2021-04-19 14:25:56 / 2021-04-19 14:25:57
            wal start/stop: 000000010000000000000012 / 000000010000000000000012
            database size: 50.6MB, database backup size: 661.5KB
            repo2: backup set size: 8MB, backup size: 28KB
            backup reference list: 20210419-142414F

The ‘walarchivemin/max’ shows the minimum and maximum WAL currently stored in the archive and, in the case of multiple repositories, will be reported across all repositories unless the --repo option is set. There may be gaps due to archive retention policies or other reasons.

Let us break the first repository by removing its content:

$ pgbackrest --stanza=demo stanza-delete --repo=1 --force
P00   INFO: stanza-delete command end: completed successfully

$ pgbackrest --stanza=demo stanza-create
P00   INFO: stanza-create for stanza demo on repo1
P00   INFO: stanza-create for stanza demo on repo2
P00   INFO: stanza demo already exists on repo2 and is valid
P00   INFO: stanza-create command end: completed successfully

If multiple repositories are configured, then a status of mixed indicates that the stanza is not in a healthy state for one or more of the repositories. In this case, the state of the stanza will be detailed in additional lines per repository.

$ pgbackrest info --stanza=demo
stanza: demo
    status: mixed
        repo1: error (no valid backups)
        repo2: ok
    cipher: none

    db (current)
        wal archive min/max (13): 00000001000000000000000E/000000010000000000000012

        full backup: 20210419-142414F
            timestamp start/stop: 2021-04-19 14:24:14 / 2021-04-19 14:24:26
            wal start/stop: 00000001000000000000000E / 00000001000000000000000E
            database size: 50.5MB, database backup size: 50.5MB
            repo2: backup set size: 8MB, backup size: 8MB

        incr backup: 20210419-142414F_20210419-142556I
            timestamp start/stop: 2021-04-19 14:25:56 / 2021-04-19 14:25:57
            wal start/stop: 000000010000000000000012 / 000000010000000000000012
            database size: 50.6MB, database backup size: 661.5KB
            repo2: backup set size: 8MB, backup size: 28KB
            backup reference list: 20210419-142414F

This state can be resolved by taking a new backup:

$ pgbackrest backup --stanza=demo --type=full --repo=1
P00   INFO: backup command begin ...
P00   INFO: execute non-exclusive pg_start_backup(): backup begins after the requested immediate checkpoint completes
P00   INFO: backup start archive = 000000010000000000000014, lsn = 0/14000028
P00   INFO: full backup size = 50.6MB
P00   INFO: execute non-exclusive pg_stop_backup() and wait for all WAL segments to archive
P00   INFO: backup stop archive = 000000010000000000000014, lsn = 0/14000138
P00   INFO: check archive for segment(s) 000000010000000000000014:000000010000000000000014
P00   INFO: new backup label = 20210419-143400F
P00   INFO: backup command end: completed successfully

Restore

Let us initiate a situation with some data, backups and restore point:

$ createdb test
$ psql -d test -c "CREATE TABLE t1(id int);"
CREATE TABLE
$ psql -d test -c "INSERT INTO t1 VALUES (1);"
INSERT 0 1

$ pgbackrest backup --stanza=demo --type=full --repo=1
P00   INFO: backup command begin ...
P00   INFO: execute non-exclusive pg_start_backup(): backup begins after the requested immediate checkpoint completes
P00   INFO: backup start archive = 000000010000000000000016, lsn = 0/16000028
P00   INFO: full backup size = 63.0MB
P00   INFO: execute non-exclusive pg_stop_backup() and wait for all WAL segments to archive
P00   INFO: backup stop archive = 000000010000000000000016, lsn = 0/16000138
P00   INFO: check archive for segment(s) 000000010000000000000016:000000010000000000000016
P00   INFO: new backup label = 20210419-143643F
P00   INFO: backup command end: completed successfully
P00   INFO: expire command begin ...
P00   INFO: repo1: expire full backup 20210419-143400F
P00   INFO: repo1: remove expired backup 20210419-143400F
P00   INFO: expire command end: completed successfully

$ psql -d test -c "INSERT INTO t1 VALUES (2);"
INSERT 0 1
$ pgbackrest backup --stanza=demo --type=full --repo=2
P00   INFO: backup command begin ...
P00   INFO: execute non-exclusive pg_start_backup(): backup begins after the requested immediate checkpoint completes
P00   INFO: backup start archive = 000000010000000000000018, lsn = 0/18000028
P00   INFO: full backup size = 63MB
P00   INFO: execute non-exclusive pg_stop_backup() and wait for all WAL segments to archive
P00   INFO: backup stop archive = 000000010000000000000018, lsn = 0/18000138
P00   INFO: check archive for segment(s) 000000010000000000000018:000000010000000000000018
P00   INFO: new backup label = 20210419-143749F
P00   INFO: backup command end: completed successfully
P00   INFO: expire command begin ...
P00   INFO: repo2: expire full backup set 20210419-142414F, 20210419-142414F_20210419-142556I
P00   INFO: repo2: remove expired backup 20210419-142414F_20210419-142556I
P00   INFO: repo2: remove expired backup 20210419-142414F
P00   INFO: expire command end: completed successfully

$ psql -d postgres -c "select pg_create_restore_point(RP1);"
 pg_create_restore_point
- ------------------------
 0/190000C8
(1 row)

$ psql -d postgres -Atc "select current_timestamp,current_setting(datestyle),txid_current();"
19-APR-21 14:38:55.313716 +00:00|Redwood, SHOW_TIME|1183

$ psql -d test -c "INSERT INTO t1 VALUES (3); SELECT pg_switch_wal();"
 pg_switch_wal
- --------------
 0/19000238
(1 row)

$ pgbackrest info --stanza=demo
stanza: demo
    status: ok
    cipher: none

    db (current)
        wal archive min/max (13): 000000010000000000000016/000000010000000000000019

        full backup: 20210419-143643F
            timestamp start/stop: 2021-04-19 14:36:43 / 2021-04-19 14:37:01
            wal start/stop: 000000010000000000000016 / 000000010000000000000016
            database size: 63.0MB, database backup size: 63.0MB
            repo1: backup set size: 10MB, backup size: 10MB

        full backup: 20210419-143749F
            timestamp start/stop: 2021-04-19 14:37:49 / 2021-04-19 14:38:03
            wal start/stop: 000000010000000000000018 / 000000010000000000000018
            database size: 63MB, database backup size: 63MB
            repo2: backup set size: 10MB, backup size: 10MB

As you can see in the example above, the backup command automatically triggered the expire command on the repository when we created a backup. Since this command is operating on one repository only, it would not expire anything in the other repositories until the backup or expire command is run for that repository.

Since the default retention policy is based on a number of backups, it is expected that only new backups are affecting the backups and archives to expire.

Restore Command

The restore command automatically defaults to selecting the latest backup from the first repository where backups exist. The order in which the repositories are checked is dictated by order of the repositories as configured in the pgbackrest.conf (e.g. repo1 will be checked before repo2). To restore from from a specific repository, the --repo option can be used.

PITR can be performed by specifying --type=time and specifying the target time with --target . If a backup is not specified via the --set option, then the configured repositories will be checked for a backup that contains the requested time. If no backup can be found, the latest backup from the first repository containing backups will be used.

$ mkdir /tmp/restored_data
$ pgbackrest restore --stanza=demo --target="RP1" --type=name --no-delta --pg1-path=/tmp/restored_data
P00   INFO: repo1: restore backup set 20210419-143643F

$ rm -rf /tmp/restored_data/*
$ pgbackrest restore --stanza=demo --target="2021-04-19 14:38:55.313716+00:00" --type=time --no-delta --pg1-path=/tmp/restored_data
HINT: time format must be YYYY-MM-DD HH:MM:SS with optional msec and optional timezone (+/- HH or HHMM or HH:MM)
    - if timezone is omitted, local time is assumed (for UTC use +00)
P00   INFO: repo1: restore backup set 20210419-143643F

$ rm -rf /tmp/restored_data/*
$ pgbackrest restore --stanza=demo --target="RP1" --type=name --no-delta --pg1-path=/tmp/restored_data --repo=2
P00   INFO: repo2: restore backup set 20210419-143749F

Even if the backup in repo2 is newer, the first found match is preserved. Let’s take a new backup in repo1 to check if pgBackRest will auto-select the backup in repo2 :

$ pgbackrest backup --stanza=demo --type=full --repo=1
P00   INFO: backup command begin ...
P00   INFO: execute non-exclusive pg_start_backup(): backup begins after the requested immediate checkpoint completes
P00   INFO: backup start archive = 00000001000000000000001B, lsn = 0/1B000028
P00   INFO: full backup size = 63MB
P00   INFO: execute non-exclusive pg_stop_backup() and wait for all WAL segments to archive
P00   INFO: backup stop archive = 00000001000000000000001B, lsn = 0/1B000138
P00   INFO: check archive for segment(s) 00000001000000000000001B:00000001000000000000001B
P00   INFO: new backup label = 20210419-144656F
P00   INFO: backup command end: completed successfully
P00   INFO: expire command begin ...
P00   INFO: repo1: expire full backup 20210419-143643F
P00   INFO: repo1: remove expired backup 20210419-143643F
P00   INFO: expire command end: completed successfully

$ pgbackrest info --stanza=demo
stanza: demo
    status: ok
    cipher: none

    db (current)
        wal archive min/max (13): 000000010000000000000018/00000001000000000000001B

        full backup: 20210419-143749F
            timestamp start/stop: 2021-04-19 14:37:49 / 2021-04-19 14:38:03
            wal start/stop: 000000010000000000000018 / 000000010000000000000018
            database size: 63MB, database backup size: 63MB
            repo2: backup set size: 10MB, backup size: 10MB

        full backup: 20210419-144656F
            timestamp start/stop: 2021-04-19 14:46:56 / 2021-04-19 14:47:10
            wal start/stop: 00000001000000000000001B / 00000001000000000000001B
            database size: 63MB, database backup size: 63MB
            repo1: backup set size: 10MB, backup size: 10MB

$ rm -rf /tmp/restored_data/*
$ pgbackrest restore --stanza=demo --target="2021-04-19 14:38:55.313716+00:00" --type=time --no-delta --pg1-path=/tmp/restored_data
P00   INFO: repo2: restore backup set 20210419-143749F

The recovery process may be complex and tricky depending on the target, and therefore, the info command can really be helpful to determine the repository to restore from.

Archive Get Command

When multiple repositories are configured, WAL will be fetched from the repositories in priority order (e.g. repo1 , repo2 , etc.). In general it is better if faster storage has higher priority. The command can operate on a single repository by specifying it with the --repo option.

root# systemctl stop edb-as-13.service
$ mv /var/lib/edb/as13/data /var/lib/edb/as13/data.orig
$ mkdir -m 700 /var/lib/edb/as13/data

$ pgbackrest restore --stanza=demo --target="2021-04-19 14:38:55.313716+00:00" --type=time --no-delta --target-action=promote
P00   INFO: restore command begin ...
P00   INFO: write updated /var/lib/edb/as13/data/postgresql.auto.conf
P00   INFO: restore global/pg_control (performed last to ensure aborted restores cannot be started)
P00   INFO: restore command end: completed successfully

$ cat /var/lib/edb/as13/data/postgresql.auto.conf|grep restore_command
restore_command = pgbackrest --stanza=demo archive-get %f "%p"

root# systemctl start edb-as-13.service
$ cat /var/lib/edb/as13/data/log/edb-2021-04-19_145147.log
...
LOG:  starting point-in-time recovery to 2021-04-19 14:38:55.313716+00
P00   INFO: found 000000010000000000000018 in the repo2: 13-1 archive
P00   INFO: found 000000010000000000000019 in the repo2: 13-1 archive
LOG:  recovery stopping before commit of transaction 1183, time 2021-04-19 14:38:55.314118+00
LOG:  redo done at 0/19000100
P00   INFO: unable to find 00000002.history in the archive
LOG:  selected new timeline ID: 2
LOG:  archive recovery complete
P00   INFO: pushed WAL file 00000002.history to the archive

$ ls /var/lib/edb/as13/backups/repo*/archive/demo/13-1/00000002.history
/var/lib/edb/as13/backups/repo1/archive/demo/13-1/00000002.history
/var/lib/edb/as13/backups/repo2/archive/demo/13-1/00000002.history

PostgreSQL found the WALs needed for recovery in our repo2 , picked a new timeline and pushed the history file to the repositories.

Whenever a new timeline is created, PostgreSQL creates a timeline history file that shows which timeline it branched off from and when. These history files are necessary to allow the system to pick the right WAL segment files when recovering from a backup that contains multiple timelines.

root# systemctl stop edb-as-13.service
$ rm -rf /var/lib/edb/as13/data/*
$ rm -rf /var/lib/edb/as13/backups/repo1/archive/demo/13-1/00000002.history
$ pgbackrest restore --stanza=demo --target="2021-04-19 14:38:55.313716+00:00" --type=time --target-timeline=current --no-delta --target-action=promote
P00   INFO: restore command begin ...
P00   INFO: repo2: restore backup set 20210419-143749F
P00   INFO: write updated /var/lib/edb/as13/data/postgresql.auto.conf
P00   INFO: restore global/pg_control (performed last to ensure aborted restores cannot be started)
P00   INFO: restore command end: completed successfully

root# systemctl start edb-as-13.service
$ cat /var/lib/edb/as13/data/log/edb-2021-04-19_145721.log
P00   INFO: found 00000002.history in the repo2: 13-1 archive
P00   INFO: unable to find 00000003.history in the archive
LOG:  selected new timeline ID: 3
P00   INFO: pushed WAL file 00000003.history to the archive

As shown above, since the history file is stored within the same repository as the restored backup set, PostgreSQL is able to pick a new and accurate timeline.

Let us now retry after moving the history file to repo1 .

root# systemctl stop edb-as-13.service
$ rm -rf /var/lib/edb/as13/data/*
$ mv /var/lib/edb/as13/backups/repo2/archive/demo/13-1/00000002.history /var/lib/edb/as13/backups/repo1/archive/demo/13-1/00000002.history
$ pgbackrest restore --stanza=demo --target="2021-04-19 14:38:55.313716+00:00" --type=time --target-timeline=current --no-delta --target-action=promote
root# systemctl start edb-as-13.service
$ cat /var/lib/edb/as13/data/log/edb-2021-04-19_150037.log
P00   INFO: found 00000002.history in the repo1: 13-1 archive
P00   INFO: found 00000003.history in the repo1: 13-1 archive
P00   INFO: unable to find 00000004.history in the archive
LOG:  selected new timeline ID: 4
P00   INFO: pushed WAL file 00000004.history to the archive

As we can see, both 00000002.history and 00000003.history have been found in repo1 so PostgreSQL could pick the next timeline correctly, even if the restored backup set came from repo2 .

When using multiple repositories, the archive-get command will tolerate gaps in one repository (due to lack of disk space e.g.) mainly because it will still be able to find the missing files in the other repository.