Monitoring

Monitoring replication setups is important to ensure that your system performs optimallyand does not run out of disk space or encounter other faults that may halt operations.

It is important to have automated monitoring in place to ensure that if, for example,replication slots start falling badly behind, the administrator is alerted and cantake proactive action.

EDB provides Postgres Enterprise Manager (PEM), which supports BDR from version 8.1.Alternatively, tools or users can make their own calls into BDR using the facilitiesdiscussed below.

Monitoring Overview

A BDR Group consists of multiple servers, often referred to as nodes. All of thenodes need to be monitored to ensure the health of the whole group.

The bdr_monitor role may execute the bdr.monitor functions to provide anassessment of BDR health using one of three levels:

  • OK - often shown as Green

  • WARNING - often shown as Yellow

  • CRITICAL - often shown as Red

  • as well as UNKNOWN - for unrecognized situations, often shown as Red

BDR also provides dynamic catalog views that show the instantaneous state of variousinternal metrics and also BDR metadata catalogs that store the configurationdefaults and/or configuration changes requested by the user. Some of those viewsand tables are accessible by bdr_monitor or bdr_read_all_stats, but some containuser or internal information that has higher security requirements.

BDR allows you to monitor each of the nodes individually, or to monitor thewhole group by access to a single node. If you wish to monitor each node individually,simply connect to each node and issue monitoring requests. If you wish to monitorthe group from a single node then use the views starting with bdr.group since theserequests make calls to other nodes to assemble a group-level information set.

If you have been granted access to the bdr.run_on_all_nodes() function bybdr_superuser then you may make your own calls to all nodes.

Monitoring Node Join and Removal

By default, the node management functions wait for the join or partoperation to complete. This can be turned off using the respectivewait_for_completion function argument. If waiting is turned off,then to see when a join or part operation finishes,check the node state indirectly via bdr.node_summary andbdr.state_journal_details .

When called, the helper function bdr.wait_for_join_completion() will causea PostgreSQL session to pause until all outstanding node join operationscomplete.

Here is an example output of a SELECT query from bdr.node_summary thatindicates that two nodes are active and another one is joining:

#  SELECT node_name, interface_connstr, peer_state_name,
#      node_seq_id, node_local_dbname
#  FROM bdr.node_summary;
- [ RECORD 1 ]-----+-----------------------------------------
node_name         | node1
interface_connstr | host=localhost dbname=postgres port=7432
peer_state_name   | ACTIVE
node_seq_id       | 1
node_local_dbname | postgres
- [ RECORD 2 ]-----+-----------------------------------------
node_name         | node2
interface_connstr | host=localhost dbname=postgres port=7433
peer_state_name   | ACTIVE
node_seq_id       | 2
node_local_dbname | postgres
- [ RECORD 3 ]-----+-----------------------------------------
node_name         | node3
interface_connstr | host=localhost dbname=postgres port=7434
peer_state_name   | JOINING
node_seq_id       | 3
node_local_dbname | postgres

Also, the table bdr.node_catchup_info will give informationon the catch-up state, which can be relevant to joining nodes or parting nodes.

When a node is parted, it could be that some nodes in the cluster did not receiveall the data from that parting node. So it will create a temporary slot froma node that already received that data and can forward it.

The catchup_state can be one of the following:

10 = setup
20 = start
30 = catchup
40 = done

Monitoring Replication Peers

There are two main views used for monitoring of replication activity:

  • bdr.node_slots for monitoring outgoing replication

  • bdr.subscription_summary for monitoring incoming replication

Most of the information provided by bdr.node_slots can be also obtained by queryingthe standard PostgreSQL replication monitoring views

Each node has one BDR group slot which should never have a connection to itand will very rarely be marked as active. This is normal, and does not implysomething is down or disconnected. See Replication Slots created by BDR .

Monitoring Outgoing Replication

There is an additional view used for monitoring of outgoing replication activity:

  • bdr.node_replication_rates for monitoring outgoing replication

The bdr.node_replication_rates view gives an overall picture of the outgoingreplication activity along with the catchup estimates for peer nodes,specifically.

#  SELECT * FROM bdr.node_replication_rates;
- [ RECORD 1 ]----+-----------
peer_node_id     | 112898766
target_name      | node1
sent_lsn         | 0/28AF99C8
replay_lsn       | 0/28AF99C8
replay_lag       | 00:00:00
replay_lag_bytes | 0
replay_lag_size  | 0 bytes
apply_rate       | 822
catchup_interval | 00:00:00
- [ RECORD 2 ]----+-----------
peer_node_id     | 312494765
target_name      | node3
sent_lsn         | 0/28AF99C8
replay_lsn       | 0/28AF99C8
replay_lag       | 00:00:00
replay_lag_bytes | 0
replay_lag_size  | 0 bytes
apply_rate       | 853
catchup_interval | 00:00:00

The apply_rate above refers to the rate in bytes per second. It is the rateat which the peer is consuming data from the local node. The replay_lag whena node reconnects to the cluster is immediately set to zero. We are working onfixing this information; as a workaround, we suggest you use the catchup_interval column that refers to the time required for the peer node to catch up to thelocal node data. The other fields are also available via the bdr.node_slots view, as explained below.

Note

This catalog is only present when bdr-enteprise extension is installed.

Administrators may query bdr.node_slots for outgoing replication from thelocal node. It shows information about replication status of all other nodesin the group that are known to the current node, as well as any additionalreplication slots created by BDR on the current node.

#  SELECT node_group_name, target_dbname, target_name, slot_name, active_pid,
#      catalog_xmin, client_addr, sent_lsn, replay_lsn, replay_lag,
#      replay_lag_bytes, replay_lag_size
#  FROM bdr.node_slots;
- [ RECORD 1 ]---+----------------------------
node_group_name | bdrgroup
target_dbname   | postgres
target_name     | node3
slot_name       | bdr_postgres_bdrgroup_node3
active_pid      | 15089
catalog_xmin    | 691
client_addr     | 127.0.0.1
sent_lsn        | 0/23F7B70
replay_lsn      | 0/23F7B70
replay_lag      | [NULL]
replay_lag_bytes| 120
replay_lag_size | 120 bytes
- [ RECORD 2 ]---+----------------------------
node_group_name | bdrgroup
target_dbname   | postgres
target_name     | node2
slot_name       | bdr_postgres_bdrgroup_node2
active_pid      | 15031
catalog_xmin    | 691
client_addr     | 127.0.0.1
sent_lsn        | 0/23F7B70
replay_lsn      | 0/23F7B70
replay_lag      | [NULL]
replay_lag_bytes| 84211
replay_lag_size | 82 kB

Note that because BDR is a mesh network, to get full view of lag in thecluster, this query has to be executed on all nodes participating.

replay_lag_bytes reports the difference in WAL positions between the localserver’s current WAL write position and replay_lsn , the last positionconfirmed replayed by the peer node. replay_lag_size is just a human-readableform of the same. It is important to understand that WAL usually contains a lotof writes that are not replicated but still count in replay_lag_bytes ,including VACUUM activity, index changes, writes associated with otherdatabases on the same node, writes for tables that are not part of areplication set, etc. So the lag in bytes reported here is not the amount ofdata that must be replicated on the wire to bring the peer node up to date,only the amount of server-side WAL that must be processed.

Similarly, replay_lag is not a measure of how long the peer node will take tocatch up, or how long it will take to replay from its current position to thewrite position at the time bdr.node_slots was queried. It measures the delaybetween when the peer confirmed the most recent commit and the currentwall-clock time. We suggest that you monitor replay_lag_bytes and replay_lag_size or catchup_interval in bdr.node_replication_rates , as this column is set tozero immediately after the node reconnects.

The lag in both bytes and time does not advance while logical replication isstreaming a transaction. It only changes when a commit is replicated. So the lagwill tend to “sawtooth”, rising as a transaction is streamed, then falling againas the peer node commits it, flushes it, and sends confirmation. The reportedLSN positions will “stair-step” instead of advancing smoothly, for similarreasons.

When replication is disconnected (active = 'f' ), the active_pid columnwill be NULL , as will client_addr and other fields that only make sensewith an active connection. The state field will be 'disconnected' . The_lsn fields will be the same as the confirmed_flush_lsn , since that is thelast position that the client is known for certain to have replayed to and saved.The _lag fields will show the elapsed time between the most recent confirmedflush on the client and the current time, and the _lag_size and _lag_bytes fields will report the distance between confirmed_flush_lsn and the localserver’s current WAL insert position.

Note: It is normal for restart_lsn to be behind the other lsn columns;this does not indicate a problem with replication or a peer node lagging. Therestart_lsn is the position that PostgreSQL’s internal logical decoding mustbe reading WAL at if interrupted, and generally reflects the position of theoldest transaction that is not yet replicated and flushed. A very oldrestart_lsn can make replication slow to restart after disconnection andforce retention of more WAL than is desirable, but will otherwise be harmless.If you are concerned, look for very long running transactions and forgottenprepared transactions.

Monitoring Incoming Replication

Incoming replication (also called subscription) can be monitored by queryingthe bdr.subscription_summary view. This shows the list of known subscriptionsto other nodes in the BDR cluster and the state of the replication worker, e.g.:

#  SELECT node_group_name, origin_name, sub_enabled, sub_slot_name,
#      subscription_status
#  FROM bdr.subscription_summary;
- [ RECORD 1 ]-------+----------------------------
node_group_name     | bdrgroup
origin_name         | node2
sub_enabled         | t
sub_slot_name       | bdr_postgres_bdrgroup_node1
subscription_status | replicating
- [ RECORD 2 ]-------+----------------------------
node_group_name     | bdrgroup
origin_name         | node3
sub_enabled         | t
sub_slot_name       | bdr_postgres_bdrgroup_node1
subscription_status | replicating

Monitoring WAL senders using LCR

If the Decoding Worker is enabled, information about thecurrent LCR (Logical Change Record ) file for each WAL sender can be monitoredvia the function bdr.wal_sender_stats ,e.g.:

postgres=# SELECT * FROM bdr.wal_sender_stats();
   pid   | is_using_lcr |       decoder_slot_name       |              lcr_file_name
- --------+--------------+-------------------------------+------------------------------------------
 2059904 | f            |                               |
 2059909 | t            | bdr_postgres_bdrgroup_decoder | 0000000000000000000000140000000000000000
 2059916 | t            | bdr_postgres_bdrgroup_decoder | 0000000000000000000000140000000000000000
(3 rows)

If is_using_lcr is FALSE , decoder_slot_name /lcr_file_name will be NULL .This will be the case if the Decoding Worker is not enabled, or the WAL sender isserving a

logical standby .

Additionally, information about the Decoding Worker can be monitored via the function bdr.get_decoding_worker_stat , e.g.:

postgres=# SELECT * FROM bdr.get_decoding_worker_stat();
   pid   | decoded_upto_lsn | waiting | waiting_for_lsn
- --------+------------------+---------+-----------------
 1153091 | 0/1E5EEE8        | t       | 0/1E5EF00
(1 row)

Monitoring BDR Replication Workers

All BDR workers show up in the system view bdr.stat_activity ,which has the same columns and information content as pg_stat_activity .So this view offers these insights into the state of a BDR system:

  • The wait_event column has enhanced information, if the reason for waiting is related to BDR.

  • The query column will be blank in BDR workers, except when a writer process is executing DDL

The bdr.workers view shows BDR worker specific details, that are notavailable from bdr.stat_activity .

The view bdr.worker_errors shows errors (if any) reported by any workerwhich has a problem continuing the work. Only active errors are visible in thisview, so if the worker was having transient problems but has recovered, theview will be empty.

Monitoring BDR Writers

There is another system view bdr.writers to monitor writer activities.This views shows the current status of only writer workers. It includes:

  • sub_name to identify the subscription which the writer belongs to

  • pid of the writer process

  • streaming_allowed to know if the writer supports application of in-progress streaming transactions

  • is_streaming to know if the writer is currently applying a streaming transaction

  • commit_queue_position to check the position of the writer in the commit queue.

BDR honours commit ordering by following the same commit order ashappened on the origin. In case of parallel writers, multiple writerscould be applying different transactions at the same time. Thecommit_queue_position shows in which order they will commit. Value 0 means that the writer is the first one to commit. Value -1 means thatthe commit position is not yet known. This can happen for a streamingtransaction or when the writer is not applying any transaction at themoment.

Monitoring Global Locks

The global lock, which is currently only used for DDL replication, is a heavyweightlock that exists across the whole BDR group.

There are currently two types of global locks:

  • DDL lock, used for serializing all DDL operations on permanent (not temporary) objects (i.e. tables) in the database

  • DML relation lock, used for locking out writes to relations during DDL operations that change the relation definition

Either or both entry types may be created for the same transaction, depending onthe type of DDL operation and the value of the bdr.ddl_locking setting.

Global locks held on the local node are visible in the ``bdr.global_locks` view </bdr/latest/catalogs#bdrglobal_locks>`__. This view shows the type of the lock; forrelation locks it shows which relation is being locked, the PID holding thelock (if local), and whether the lock has been globally granted or not. In caseof global advisory locks, lock_type column shows GLOBAL_LOCK_ADVISORY andrelation column shows the advisory key(s) on which the lock is acquired.

The following is an example output of bdr.global_locks while running anALTER TABLE statement with bdr.ddl_locking = on :

#  SELECT lock_type, relation, pid FROM bdr.global_locks;
- [ RECORD 1 ]--------------
lock_type | GLOBAL_LOCK_DDL
relation  | [NULL]
pid       | 15534
- [ RECORD 2 ]--------------
lock_type | GLOBAL_LOCK_DML
relation  | someschema.sometable
pid       | 15534

See the catalog documentation for details on all fields including locktiming information.

Monitoring Conflicts

Replication Monitoring Conflicts can arise when multiple nodes makechanges that affect the same rows in ways that can interact with each other.The BDR system should be monitored to ensure that conflicts are identifiedand, where possible, application changes are made to eliminate them or makethem less frequent.

By default, all conflicts are logged to bdr.conflict_history . Since thiscontains full details of conflicting data, the rows are protected byrow-level security to ensure they are visible only byowners of replicated tables. Owners should expect conflicts and analyze themto see which, if any, might be considered as problems to be resolved.

For monitoring purposes use bdr.conflict_history_summary , which doesnot contain user data. An example query to count the number of conflictsseen within the current day using an efficient query plan is:

SELECT count(*)
FROM bdr.conflict_history_summary
WHERE local_time > date_trunc(day, current_timestamp)
  AND local_time < date_trunc(day, current_timestamp + 1 day);

External Monitoring

User supplied metadata can be stored to allow monitoring tools to understandand monitor the BDR cluster. By centralizing this information, externaltools can access any single node and read details about the wholecluster, such as network cost and warning/alarm thresholds for specificconnections.

bdr_superuser has the privileges on these functions and tables.The view bdr.network_monitoring is also accessible by thebdr_read_all_stats role.

bdr.set_node_location

This function inserts node metadata into bdr.node_location

Synopsis

bdr.set_node_location(
    node_group_name text,
    node_name text,
    node_region text,
    node_location text);

Parameters

  • node_group_name - name of the BDR group

  • node_name - name of the node

  • node_region - the datacenter site or Region

  • node_location - the server name, availability zone etc..

bdr.set_network_path_info

This function inserts network path metadata for network paths between nodesinto the table bdr.network_path_info .

Synopsis

bdr.set_network_path_info(
    node_group_name text,
    region1 text,
    region2 text,
    location1 text,
    location2 text,
    network_cost numeric,
    warning_threshold numeric,
    alarm_threshold numeric)

Parameters

  • node_group_name - name of the BDR group

  • region1 - the origin server name

  • region2 - the remote server name

  • location1 - the origin datacente name

  • location2 - the remote datacenter name

  • network_cost - an abstract value representing the cost of network transfer

  • warning_threshold - a delay above which a threshold should be raised

  • alarm_threshold - a delay above which an alarm should be raised

bdr.network_monitoring view

This view collects information about the network path between nodes.

The configuration of logging is defined by the bdr.alter_node_set_log_config function.

Apply Statistics

BDR collects statistics about replication apply, both for each subscriptionand for each table.

Two monitoring views exist: bdr.stat_subscription for subscription statisticsand bdr.stat_relation for relation statistics. These views both provide:

  • Number of INSERTs/UPDATEs/DELETEs/TRUNCATEs replicated

  • Block accesses and cache hit ratio

  • Total I/O time for read/write

  • Number of in-progress transactions streamed to file

  • Number of in-progress transactions streamed to writers

  • Number of in-progress streamed transactions committed/aborted

and for relations only, these statistics:

  • Total time spent processing replication for the relation

  • Total lock wait time to acquire lock (if any) for the relation (only)

and for subscriptions only, these statistics:

  • Number of COMMITs/DDL replicated for the subscription

  • Number of times this subscription has connected upstream

Tracking of these statistics is controlled by the BDR GUCsbdr.track_subscription_apply and bdr.track_relation_apply respectively.

The example output from these would look like this:

#  SELECT sub_name, nconnect, ninsert, ncommit, nupdate, ndelete, ntruncate, nddl
FROM bdr.stat_subscription;
- [ RECORD 1 ]----------------------------------
sub_name  | bdr_regression_bdrgroup_node1_node2
nconnect  | 3
ninsert   | 10
ncommit   | 5
nupdate   | 0
ndelete   | 0
ntruncate | 0
nddl      | 2

In this case the subscription connected 3 times to the upstream, inserted10 rows and did 2 DDL commands inside 5 transactions.

Stats counters for these views can be reset to zero using the functionsbdr.reset_subscription_stats and bdr.reset_relation_stats .

Standard PostgreSQL Statistics Views

Statistics on table and index usage are updated normally by the downstreammaster. This is essential for the correct function of

autovacuum .If there are no local writes on the downstream master

and statistics have not beenreset, these two views should show corresponding results between upstream and downstream:

  • pg_stat_user_tables

  • pg_statio_user_tables

Note

We don’t necessarily expect the upstream table statistics to be similar to the downstream ones; we only expect them to change by the same amounts. Consider the example of a table whose statistics show 1M inserts and 1M updates; when a new node joins the BDR group, the statistics for the same table in the new node will show 1M inserts and zero updates. However, from that moment, the upstream and downstream table statistics will change by the same amounts, because all changes on one side will be replicated to the other side.

Since indexes are used to apply changes, the identifying indexes on thedownstream side may appear more heavily used with workloads that performUPDATE s and DELETE s than non-identifying indexes are.

The built-in index monitoring views are:

  • pg_stat_user_indexes

  • pg_statio_user_indexes

All these views are discussed in detail in the PostgreSQL documentation on the statistics views .

Monitoring BDR Versions

BDR allows running different Postgres versions as well as differentBDR versions across the nodes in the same cluster. This is useful forupgrading.

The view bdr.group_versions_details uses the functionbdr.run_on_all_nodes() to retrieve Postgres and BDR versions from all nodesat the same time. For example:

bdrdb=# SELECT node_name, postgres_version, bdr_version
        FROM bdr.group_versions_details;
 node_name | postgres_version | bdr_version
- ----------+------------------+-------------
 node1     | 14.1             | 4.0.0
 node2     | 14.1             | 4.0.0

The recommended setup is to try to have all nodes running the samelatest versions as soon as possible. It is recommendedthat the cluster does not run different BDR versions for too long.

For monitoring purposes, we recommend the following alert levels:

  • status=UNKNOWN, message=This node is not part of any BDR group

  • status=OK, message=All nodes are running same BDR versions

  • status=WARNING, message=There is at least 1 node that is not accessible

  • status=WARNING, message=There are node(s) running different BDR versions when compared to other nodes

The described behavior is implemented in the functionbdr.monitor_group_versions() , which uses BDR versioninformation returned from the view bdr.group_version_details to provide a cluster-wide version check. For example:

bdrdb=# SELECT * FROM bdr.monitor_group_versions();
 status |                message
- -------+-----------------------------------------
 OK     | All nodes are running same BDR versions

Monitoring Raft Consensus

Raft Consensus should be working cluster-wide at all times. The impactof running a BDR cluster without Raft Consensus working might be asfollows:

  • BDR data changes replication may still be working correctly

  • Global DDL/DML locks will not work

  • Galloc sequences will eventually run out of chunks

  • Eager Replication will not work

  • Cluster maintenance operations (join node, part node, promote standby) are still allowed but they might not finish (simply hang)

  • Node statuses might not be correctly synced among the BDR nodes

  • BDR group replication slot does not advance LSN, thus keeps WAL files on disk

The view bdr.group_raft_details uses the functionsbdr.run_on_all_nodes() and bdr.get_raft_status() to retrieve RaftConsensus status from all nodes at the same time. For example:

bdrdb=# SELECT node_id, node_name, state, leader_id
FROM bdr.group_raft_details;
  node_id   | node_name |     state     | leader_id
- -----------+-----------+---------------+------------
 1148549230 | node1     | RAFT_LEADER   | 1148549230
 3367056606 | node2     | RAFT_FOLLOWER | 1148549230

We can say that Raft Consensus is working correctly if all belowconditions are met:

  • A valid state (RAFT_LEADER or RAFT_FOLLOWER ) is defined on all nodes

  • Only one of the nodes is the RAFT_LEADER

  • The leader_id is the same on all rows and must match the node_id of the row where state = RAFT_LEADER

From time to time, Raft Consensus will start a new election to define anew RAFT_LEADER . During an election, there might be an intermediarysituation where there is no RAFT_LEADER and some of the nodes considerthemselves as RAFT_CANDIDATE . The whole election should not take longerthan bdr.raft_election_timeout (by default it is set to 6 seconds). Ifthe query above returns an in-election situation, then simply wait forbdr.raft_election_timeout and run the query again. If afterbdr.raft_election_timeout has passed and some the conditions above arestill not met, then Raft Consensus is not working.

Raft Consensus might not be working correctly on a single node only;for example one of the nodes does not recognize the current leader andconsiders itself as a RAFT_CANDIDATE . In this case, it is important tomake sure that:

  • All BDR nodes are accessible to each other through both regular and replication connections (check file pg_hba.conf )

  • BDR versions are the same on all nodes

  • bdr.raft_election_timeout is the same on all nodes

In some cases, especially if nodes are geographically distant from eachother and/or network latency is high, the default value ofbdr.raft_election_timeout (6 seconds) might not be enough. If RaftConsensus is still not working even after making sure everything iscorrect, consider increasing bdr.raft_election_timeout to, say, 30seconds on all nodes. From BDR 3.6.11 onwards, settingbdr.raft_election_timeout requires only a server reload.

Given how Raft Consensus affects cluster operational tasks, and also asRaft Consensus is directly responsible for advancing the group slot,we can define monitoring alert levels as follows:

  • status=UNKNOWN, message=This node is not part of any BDR group

  • status=OK, message=Raft Consensus is working correctly

  • status=WARNING, message=There is at least 1 node that is not accessible

  • status=WARNING, message=There are node(s) as RAFT_CANDIDATE, an election might be in progress

  • status=WARNING, message=There is no RAFT_LEADER, an election might be in progress

  • status=CRITICAL, message=There is a single node in Raft Consensus

  • status=CRITICAL, message=There are node(s) as RAFT_CANDIDATE while a RAFT_LEADER is defined

  • status=CRITICAL, message=There are node(s) following a leader different than the node set as RAFT_LEADER

The described behavior is implemented in the functionbdr.monitor_group_raft() , which uses Raft Consensus statusinformation returned from the view bdr.group_raft_details to provide a cluster-wide Raft check. For example:

bdrdb=# SELECT * FROM bdr.monitor_group_raft();
 status |               message
- -------+-------------------------------------
 OK     | Raft Consensus is working correctly

Monitoring Replication Slots

Each BDR node keeps:

  • One replication slot per active BDR peer

  • One group replication slot

For example:

bdrdb=# SELECT slot_name, database, active, confirmed_flush_lsn
FROM pg_replication_slots ORDER BY slot_name;
        slot_name         | database | active | confirmed_flush_lsn
- -------------------------+----------+--------+---------------------
 bdr_bdrdb_bdrgroup       | bdrdb    | f      | 0/3110A08
 bdr_bdrdb_bdrgroup_node2 | bdrdb    | t      | 0/31F4670
 bdr_bdrdb_bdrgroup_node3 | bdrdb    | t      | 0/31F4670
 bdr_bdrdb_bdrgroup_node4 | bdrdb    | t      | 0/31F4670

Peer slot names follow the convention bdr_<DATABASE>_<GROUP>_<PEER> ,while the BDR group slot name follows the conventionbdr_<DATABASE>_<GROUP> , which can be accessed using the functionbdr.local_group_slot_name() .

Peer replication slots should be active on all nodes at all times.If a peer replication slot is not active, then it might mean:

  • The corresponding peer is shutdown or not accessible; or

  • BDR replication is broken.

Grep the log file for ERROR or FATAL and also check bdr.worker_errors onall nodes. The root cause might be, for example, an incompatible DDL wasexecuted with DDL replication disabled on one of the nodes.

The BDR group replication slot is however inactive most of the time. BDRmaintains this slot and advances its LSN when all other peers have alreadyconsumed the corresponding transactions. Consequently it is not necessary tomonitor the status of the group slot.

The function bdr.monitor_local_replslots() provides a summary of whether allBDR node replication slots are working as expected, e.g.:

bdrdb=# SELECT * FROM bdr.monitor_local_replslots();
 status |                    message
- -------+-------------------------------------------------
 OK     | All BDR replication slots are working correctly

One of the following status summaries will be returned:

  • UNKNOWN : This node is not part of any BDR group

  • OK : All BDR replication slots are working correctly

  • OK : This node is part of a subscriber-only group

  • CRITICAL : There is at least 1 BDR replication slot which is inactive

  • CRITICAL : There is at least 1 BDR replication slot which is missing

Monitoring Transaction COMMITs

By default, BDR transactions commit only on the local node. In that case,transaction COMMIT will be processed quickly.

BDR can be used with standard PostgreSQL synchronous replication, whileBDR also provides two new transaction commit modes: CAMO and Eagerreplication. Each of these modes provides additional robustnessfeatures, though at the expense of additional latency at COMMIT .The additional time at COMMIT can be monitored dynamically using thebdr.stat_activity catalog, where processes report different wait_event states. A transaction in COMMIT waiting for confirmations from one ormore synchronous standbys reports a SyncRep wait event, whereas thetwo new modes report EagerRep .