Commit scopes
=============

Commit scopes give applications granular control about durability and
consistency of EDB Postgres Distributed.

A commit scope is a named rule that describes the behavior of COMMIT
replication. The actual behavior depends on whether a commit scope uses
 :ref:`Group commit <Group commit>`  , :ref:`Commit At Most Once <Commit At Most Once>`  , :ref:`Lag Control <Lag Control>`  , or combination of these.

Configuration
-------------

To use Group Commit, first define a commit scope, which determines the
PGD nodes involved in the commit of a transaction. Once a scope is
established, you can configure a transaction to use Group Commit as
follows:

.. code:: sql

   BEGIN;
   SET LOCAL bdr.commit_scope = example_scope;
   ...
   COMMIT;

You must set the commit scope before the transaction writes any data.

For this example, you might previously have defined the commit scope as:

.. code:: sql

   SELECT bdr.add_commit_scope(
       commit_scope_name := example_scope,
       origin_node_group := example_bdr_group,
       rule := ANY 2 (example_bdr_group) GROUP COMMIT,
       wait_for_ready := true
   );

This example assumes a *node group* named ``example_bdr_group`` exists
and includes at least two PGD nodes as members, either directly or in
subgroups. Any transaction committed in the ``example_scope`` requires
one extra confirmation from a PGD node in the group. Together with the
origin node, this accounts for “ANY 2” nodes out of the group, on which
the transaction is guaranteed to be visible and durable after the
commit.

Origin groups
^^^^^^^^^^^^^

Rules for commit scopes can depend on the node the transaction is
committed on, that is, the node that acts as the origin for the
transaction. To make this transparent for the application, PGD allows a
commit scope to define different rules depending on where the
transaction originates from.

For example, consider an EDB Postgres Distributed cluster with nodes
spread across two data centers: a left and a right one. Assume the
top-level PGD node group is called ``top_group`` . You can use the
following commands to set up subgroups and create a commit scope
requiring all nodes in the local data center to confirm the transaction
but only one node from the remote one:

.. code:: sql

   - - create sub-groups
   SELECT bdr.create_node_group(
       node_group_name := left_dc,
       parent_group_name := top_group,
       join_node_group := false
   );
   SELECT bdr.create_node_group(
       node_group_name := right_dc,
       parent_group_name := top_group,
       join_node_group := false
   );

   - - create a commit scope with individual rules
   - - for each sub-group
   SELECT bdr.add_commit_scope(
       commit_scope_name := example_scope,
       origin_node_group := left_dc,
       rule := ALL (left_dc) GROUP COMMIT AND ANY 1 (right_dc) GROUP COMMIT,
       wait_for_ready := true
   );
   SELECT bdr.add_commit_scope(
       commit_scope_name := example_scope,
       origin_node_group := right_dc,
       rule := ANY 1 (left_dc) GROUP COMMIT AND ALL (right_dc) GROUP COMMIT,
       wait_for_ready := true
   );

Now, using the ``example_scope`` on any node that’s part of ``left_dc``
uses the first scope. Using the same scope on a node that’s part of
``right_dc`` uses the second scope. This is an effective way of creating
an inverted scope without having to juggle scope names in the
application.

Each group can also have a default commit scope specified using the
``bdr.alter_node_group_option`` admin interface.

Making the above scopes the default ones for all transactions
originating on nodes in those groups looks like this:

.. code:: sql

   SELECT bdr.alter_node_group_option(
     node_group_name := left_dc,
     config_key := default_commit_scope,
     config_value := example_scope
   );
   SELECT bdr.alter_node_group_option(
     node_group_name := right_dc,
     config_key := default_commit_scope,
     config_value := example_scope
   );

Confirmation levels
^^^^^^^^^^^^^^^^^^^

PGD nodes can send confirmations for a transaction at different times.
In increasing levels of protection, from the perspective of the
confirming node, these are:

-  ``received`` — A remote PGD node confirms the transaction immediately
   after receiving it, prior to starting the local application.

-  ``replicated`` — Confirms after applying changes of the transaction
   but before flushing them to disk.

-  ``durable`` — Confirms the transaction after all of its changes are
   flushed to disk.

-  ``visible`` (default) — Confirms the transaction after all of its
   changes are flushed to disk and it’s visible to concurrent
   transactions.

In rules for commit scopes, you can append these confirmation levels to
the node group definition in parentheses with ``ON`` as follows:

-  ``ANY 2 (right_dc) ON replicated``

-  ``ALL (left_dc) ON visible`` (default)

-  ``ALL (left_dc) ON received AND ANY 1 (right_dc) ON durable``

Reference
---------

Commit scope grammar
^^^^^^^^^^^^^^^^^^^^

The grammar for commit scopes is composed as follows:

::

   commit_scope:
       confirmation [AND ...]

   commit_scope_operation:
       commit_scope_group [ ON { received|replicated|durable|visible } ] commit_scope_kind

   commit_scope_group:
   { ANY num (node_group [, ...])
     | MAJORITY (node_group [, ...])
     | ALL (node_group [, ...]) }

   commit_scope_kind:
   { GROUP COMMIT [ ( group_commit_parameter = value ) ] [ ABORT ON ( abort_on_parameter = value ) ]
     | CAMO [ DEGRADE ON ( degrade_on_parameter = value ) TO ASYNC ]
     | LAG CONTROL [ ( lag_control_parameter = value ) ] }

Parameters
^^^^^^^^^^

-  ``node_group`` — Name of a node group.

-  ``( group_commit_parameter = value )`` — Options for Group Commit. \*
   ``transaction_tracking`` (Boolean) — Specifies whether to track
   status of transaction. \* ``conflict_resolution`` (enum) — Specifies
   how to handle conflicts. Possible values are ``async`` , meaning to
   resolve conflicts asynchronously during replication using the
   conflict resolution policy, or ``eager`` meaning that conflicts are
   resolved eagerly during COMMIT by aborting one of the conflicting
   transactions. \* ``commit_decision`` (enum) — Specifies how the
   COMMIT decision is made. The value ``group`` means the
   ``commit_scope_group`` specification also affects the COMMIT
   decision, not just durability. The value ``partner`` means the
   partner node decides whether transactions can be committed. This
   value is allowed only on 2 data node groups. The value ``raft`` means
   the COMMIT decision is done using Raft consensus independent of
   ``commit_scope_group`` consensus.

-  ``ABORT ON ( abort_on_parameter = value )`` — Allows automatic
   transaction abort on timeout. \* ``timeout`` (interval) — Timeout in
   milliseconds (accepts other units).

-  ``DEGRADE ON ( degrade_on_parameter = value )`` — Allows degrading to
   asynchronous operation on timeout \* ``timeout`` (interval) — Timeout
   in milliseconds (accepts other units) after which operation becomes
   asynchronous. \* ``require_write_lead`` (Boolean) — Specifies whether
   the node must be a write lead to be able to switch to asynchronous
   mode.

-  ``( lag_control_parameter = value )`` — Options for Lag Control: \*
   ``max_lag_size`` (int) — Maximum allowed lag based on WAL bytes. \*
   ``max_lag_time`` (interval) — Maximum allowed lag based on wall clock
   sampling. \* ``max_commit_delay`` (interval) — Maximum delay that can
   be injected to commit to try to keep within the lag limits.

..  Note::
   `CAMO`  commit scope kind is mostly syntactic sugar for `GROUP COMMIT (transaction_tracking = true, commit_decision = partner)`  with additional `DEGRADE ON`  clause. It's expected that `GROUP COMMIT`  will eventually gain the `DEGRADE ON`  clause as well, making `CAMO`  syntax deprecated.

..  Note::
   While the grammar for `synchronous_standby_names`  and commit scopes can look similar, the former doesn't account for the origin node, but the latter does. Therefore, for example `synchronous_standby_names = 'ANY 1 (..)'`  is equivalent to a commit scope of `ANY 2 (...)` .  This choice makes reasoning about majority easier and reflects that the origin node also contributes to the durability and visibility of the transaction.

Adding a commit scope rule
^^^^^^^^^^^^^^^^^^^^^^^^^^

The function ``bdr.add_commit_scope`` creates a rule for the given
commit scope name and origin node group. If the rule is the same for all
nodes in the EDB Postgres Distributed cluster, invoking this function
once for the top-level node group is enough to fully define the commit
scope.

Alternatively, you can invoke it multiple times with the same
``commit_scope_name`` but different origin node groups and rules for
commit scopes that vary depending on the origin of the transaction.

Synopsis
^^^^^^^^

.. code:: sql

   bdr.add_commit_scope(
       commit_scope_name NAME,
       origin_node_group NAME,
       rule TEXT
       wait_for_ready boolean DEFAULT true)

Changing a commit scope rule
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

To change a specific rule for a single origin node group in a commit
scope, you can use the function ``bdr.alter_commit_scope`` .

.. _synopsis-1:

Synopsis
^^^^^^^^

.. code:: sql

   bdr.alter_commit_scope(
       commit_scope_name NAME,
       origin_node_group NAME,
       rule TEXT)

Removing a commit scope rule
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

You can use ``bdr.remove_commit_scope`` to drop a single rule in a
commit scope. If you define multiple rules for the commit scope, you
must invoke this function once per rule to fully remove the entire
commit scope.

.. _synopsis-2:

Synopsis
^^^^^^^^

.. code:: sql

   bdr.remove_commit_scope(
       commit_scope_name NAME,
       origin_node_group NAME)

..  Note::
   Removing a commit scope that's still used as default by a node group isn't allowed.