Parallel Apply ============== What is Parallel Apply? ----------------------- Parallel Apply is a feature of PGD that allows a PGD node to use multiple writers per subscription. This behavior generally increases the throughput of a subscription and improves replication performance. Configuring Parallel Apply ^^^^^^^^^^^^^^^^^^^^^^^^^^ Two variables control Parallel Apply in PGD: `bdr.max_writers_per_subscription `_ (defaults to 8) and `bdr.writers_per_subscription `_ (defaults to 2). .. code:: plain bdr.max_writers_per_subscription = 8 bdr.writers_per_subscription = 2 This configuration gives each subscription two writers. However, in some circumstances, the system might allocate up to eight writers for a subscription. Changing `bdr.max_writers_per_subscription `_ requires a server restart to take effect. You can change `bdr.writers_per_subscription `_ for a specific subscription without a restart by: 1. Halting the subscription using `bdr.alter_subscription_disable `_ . 2. Setting the new value. 3. Resuming the subscription using `bdr.alter_subscription_enable `_ . First though, establish the name of the subscription using ``select * from bdr.subscription`` . For this example, the subscription name is ``bdr_pgddb_bdrgroup_node2_node1`` . .. code:: sql SELECT bdr.alter_subscription_disable (bdr_pgddb_bdrgroup_node2_node1); UPDATE bdr.subscription SET num_writers = 4 WHERE sub_name = bdr_pgddb_bdrgroup_node2_node1; SELECT bdr.alter_subscription_enable (bdr_pgddb_bdrgroup_node2_node1); When to use Parallel Apply ^^^^^^^^^^^^^^^^^^^^^^^^^^ Parallel Apply is always on by default and, for most operations, we recommend leaving it on. Monitoring Parallel Apply ^^^^^^^^^^^^^^^^^^^^^^^^^ To support Parallel Apply’s deadlock mitigation, PGD adds columns to `bdr.stat_subscription `_ . The new columns are ``nprovisional_waits`` , ``ntuple_waits`` , and ``ncommmit_waits`` . These are metrics that indicate how well Parallel Apply is managing what previously would have been deadlocks. They don’t reflect overall system performance. The ``nprovisional_waits`` value reflects the number of operations on the same tuples being performed by concurrent apply transactions. These are provisional waits that aren’t actually waiting yet but could start waiting. If a tuple’s write needs to wait until it can be safely applied, it’s counted in ``ntuple_waits`` . Fully applied transactions that waited before being committed are counted in ``ncommit_waits`` . Disabling Parallel Apply ^^^^^^^^^^^^^^^^^^^^^^^^ To disable Parallel Apply, set `bdr.writers_per_subscription `_ to ``1`` . Deadlock mitigation ^^^^^^^^^^^^^^^^^^^ When Parallel Apply is operating, the transactional changes from the subscription are written by multiple writers. However, each writer ensures that the final commit of its transaction doesn’t violate the commit order as executed on the origin node. If there’s a violation, an error is generated and the transaction can be rolled back. This mechanism previously meant that when the following are all true, the resulting error could manifest as a deadlock: - A transaction is pending commit and modifies a row that another transaction needs to change. - That other transaction executed on the origin node before the pending transaction did. - The pending transaction took out a lock request. Additionally, handling the error could increase replication lag due to a combination of the time taken: - To detect the deadlock - For the client to roll back its transaction - For indirect garbage collection of the changes that were already applied - To redo the work This is where Parallel Apply’s deadlock mitigation can help. For any transaction, Parallel Apply looks at transactions already scheduled for any row (tuple) that the current transaction wants to write. If it finds one, the row is marked as needing to wait until the other transaction is committed before applying its change to the row. This approach ensures that rows are written in the correct order. Parallel Apply support ^^^^^^^^^^^^^^^^^^^^^^ In PGD 6, Parallel Apply works with CAMO and Quorum Commit. It isn’t compatible with Group Commit. You must disable Parallel Apply if Group Commit is in use.