pgd node setup ============== Synopsis -------- The ``pgd node setup`` command is used to configure PGD data nodes in a cluster. It can be used to set up a new node, join an existing node to a cluster, or perform a logical join of a node to the cluster. .. Important "Version requirement for physical joins":: When `pgd node setup` performs a physical join (copying data from the remote node when the local node isn't up and running), it requires that both the source node and the joining node have exactly the same PGD version. You can't use physical joins to join a node with a different PGD version to an existing cluster. For rolling upgrades, ensure you use logical joins instead. See :ref:`Rolling upgrade using node join ` for more details. .. :: When initializing a new node in single-user mode, the command checks whether the `NextOID` counter is in the user range (≥ 16384). If not, it advances the counter using `pg_resetwal` before creating BDR objects (database, extension, schema). Advancing the counter ensures those objects are assigned user-range OIDs, preventing pre-flight check failures when later running `pgd node upgrade` . The behavior of the command depends on the state of the local node and the remote node specified in the command. If this is the first node in the cluster, ``pgd node setup`` will perform ``initdb`` and setup PGD node. If this is not the first node, but the local node is not up and running, ``pgd node setup`` will perform a physical join of the node to the cluster. This will copy the data from the remote node to the local node as part of the initialization process, then join the local node to the cluster. This is the fastest way to load data into a new node. On a large database, replaying the write-ahead log generated during the base backup can take a long time. By default, ``pgd node setup`` waits indefinitely for this replay and the node’s promotion to complete, and a retry resumes from where a previous attempt left off rather than redoing the base backup. Set ``--timeout`` (or the ``PGCTLTIMEOUT`` environment variable) to bound this wait instead. For more control over the process, such as managing the base backup and its replication slot yourself, see `Joining large PGD nodes with 2 steps `_ . If the local node is up and running and remote node also is reachable, ``pgd node setup`` will perform a logical join of the node to the cluster. This will create a new node in the cluster and start streaming replication from the remote node. This is the recommended way to add a new node to an existing cluster. If the local node is up and running and remote node dsn is not provided, ``pgd node setup`` will do a node group switch if node not part of the given group. Users and roles ^^^^^^^^^^^^^^^ The ``pgd node setup`` command requires a superuser role to run. The superuser role is used to create the data directory and initialize the database. The superuser role must have the ``CREATEDB`` privilege to create the database. The user specified in the ``--dsn`` option will be created if it does not exist. It will only be granted the ``bdr_superuser`` role which will allow it to administer PGD functionality. It will not, though have any other privileges on the database. See :ref:`User roles ` for the full PGD role hierarchy. Support for Transparent Data Encryption (TDE) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Transparent Data Encryption (TDE) is an optional feature available in EDB Postgres Advanced Server and EDB Postgres Extended Server versions 15 and later. The ``pgd node setup`` command supports this feature when initializing a new database server or when joining an existing node to the cluster. The TDE options exposed by ``pgd node setup`` are similar to the `initdb TDE options `_ , with the following exceptions: - The option ``--data-encryption`` is a boolean-only flag (it does not accept the AES key length value). - Instead, use the option ``--data-encryption-keylen`` to specify the AES key length. .. Important "Key-wrapping choice is required":: When you enable TDE with `--data-encryption` , `pgd node setup` requires an explicit key-wrapping choice. Provide `--key-wrap-command` and `--key-unwrap-command` , or set the equivalent `PGDATAKEYWRAPCMD` and `PGDATAKEYUNWRAPCMD` environment variables, to wrap the data encryption key. Otherwise, pass `--no-key-wrap` to store it unwrapped, which isn't recommended for production use. Without one of these options, the command fails immediately with an error instead of defaulting to no wrapping. .. :: ## Syntax .. code:: plaintext pgd node setup [OPTIONS] -D Arguments --------- - ```` The name of the node to be created. This is the name that will be used to identify the node in the cluster. It must be unique within the cluster. Options ------- .. csv-table:: :header: Option                                             ,Description :widths: 10,30 :align: left :class: longtable `--listen-addr `,"The address that the configured node will listen on for incoming connections, and the address that other nodes will use to connect to this node. This is typically set to at least `localhost`, but can be set to any valid address. The default is `localhost`. The `host` value from the `--dsn` will also be appended to this list." `--initial-node-count `,Number of nodes in the cluster (or planned to be in the cluster). Used to calculate various resource settings for the node. Default is 3. `--bindir `,<BINDIR> Specifies the directory where the binaries are located. Defaults to the directory where the running pgd binary is located. `--log-file `,"Path to log file, used for postgres startup logs. Default is to write to a file in the current directory named `postgres-.log` where the port value is fetched from the `port` attribute of `--dsn` option." "`-D`, `--pgdata `",Uses <PG_DATA> as the data directory of the node. (Also set with environment variable `PGDATA`). It must be a valid directory and must be writable by the user running the command. `--superuser `,Superuser name for `initdb`. Default is `postgres`. `--node-kind `,"Specifies the kind of node to be created. Default is `data`. Supported values are `data`, `witness`, `subscriber-only`." `--group-name `,"Node group name. If not provided, the node will be added to the group of the active node. It is a mandatory argument for the first node of a group." `--create-group`,"Set this flag to create the given group, if it is not already present. This will be true by default for the first node." `--cluster-name `,Name of the cluster to join the node to. When setting up cluster for the first time this will be used to create the `parent node group`. Defaults to `pgd` if not specified. `--cluster-dsn `,"A DSN which belongs to the active PGD cluster. This is not required when configuring the first node of a cluster, however is mandatory for subsequent nodes. Should point to the DSN of an existing active node." `--postgresql-conf `,Optional path of the `postgresql.conf` file to be used for the node. `--postgresql-auto-conf `,Optional path of the `postgresql.auto.conf` file to be used for the node. `--hba-conf `,Optional path of the `pg_hba.conf` file to be used for the node. `--update-pgpass`,"If set, the pgpass file for the new nodes password will be stored in the current user's `.pgpass` file." `--verbose`,Print verbose messages. "`-y`, `--data-encryption`","Adds TDE when initializing a database server. Requires an explicit key-wrapping choice, either `--key-wrap-command` and `--key-unwrap-command`, or `--no-key-wrap`." `--data-encryption-keylen `,The AES key length for TDE. Default is `128`. Supported values are `128` and `256`. `--key-wrap-command `,"Used with `--data-encryption` (`-y`). The wrapping/encryption command to protect the data encryption key. `` is customizable, but it must contain the placeholder `%p`.
If you don't use this option, `pgd node setup` falls back to the environment variable `PGDATAKEYWRAPCMD`." `--key-unwrap-command `,"Used with `--data-encryption` (`-y`). The unwrapping/decryption command to access the data encryption key. `` is customizable, but it must contain the placeholder `%p`.
If you don't use this option, `pgd node setup` falls back to the environment variable `PGDATAKEYUNWRAPCMD`." `--no-key-wrap`,Used with `--data-encryption` (`-y`). Disable the key wrapping. This option is not recommended for production use. Required if you don't provide `--key-wrap-command` and `--key-unwrap-command`. `--copy-key-from `,"Copy an existing data encryption key from the provided location.
Normally, encryption keys are stored in `pg_encryption/key.bin`" See also `Global Options `_ . The ``pgd node setup`` command attempts to find the database password non-interactively by checking the following methods of retrieval in order of precedence: - Password set with the ``password`` parameter provided with the ``--dsn`` global option. - Password set with the ``PGPASSWORD`` environment variable. - Password file set with the ``passfile`` parameter provided with the ``--dsn`` global option. - Password file set with the ``PGPASSFILE`` environment variable. - Password file ``.pgpass`` in the ``HOME`` directory of the user invoking the command. If the system can’t find a password via any of these methods, the command prompts you to enter the password manually. The ``--update-pgpass`` flag controls what happens after the system has successfully obtained a password. If set, it ensures that the password is saved for future non-interactive use. The password file to be updated follows this precedence order: - The path specified by the ``passfile`` parameter within the connection string provided for the ``--dsn option`` . - The file specified by the ``PGPASSFILE`` environment variable. - The ``.pgpass`` file located in the ``HOME`` directory of the user invoking the command. Examples -------- In these examples, we will set up a cluster with on three hosts, ``host-1`` , ``host-2`` and ``host-3`` , to create three nodes: ``node-1`` , ``node-2`` , and ``node-3`` . The three nodes will be data nodes, and part of a cluster named ``pgd`` with the group name ``group-1`` . We recommend that you export the PGPASSWORD environment variable to avoid having to enter the password for the ``pgdadmin`` user each time you run a command. You can do this with the following command: .. code:: shell export PGPASSWORD=pgdsecret Configuring the first node ^^^^^^^^^^^^^^^^^^^^^^^^^^ .. code:: shell pgd node node-1 setup --dsn "host=host-1 port=5432 user=pgdadmin dbname=pgddb" \ - -listen-addr "localhost,host-1" \ - -group-name group-1 --cluster-name pgd \ - D /var/lib/edb-pge/17/main Stepping through the command, we are setting up ``node-1`` . The first option is the ``--dsn`` option, which is the connection string for the node. This is typically set to ``host=hostname port=5432 user=pgdadmin dbname=pgd`` , which is a typical connection string for a local Postgres instance. The ``--listen-address`` option is used to specify the address that the node will listen on for incoming connections. In this case, we are setting it to ``localhost,host-1`` , which means that the node will listen on both the localhost and the ``host-1`` address. This is the first node in the cluster, so we set the group name to ``group-1`` and the cluster name to ``pgd`` (which is actually the default). As this is the first node in the cluster, the ``--create-group`` option is automatically set. Finally, we set the data directory for the node with the ``-D`` option; this is where the Postgres data files will be stored. In this example, we are using ``/var/lib/edb-pge/17/main`` as the data directory. The command will create the data directory and initialize the database correctly for PGD. It will then start the node and make it available for new connections, including the other nodes joining the cluster. Configuring a second node ^^^^^^^^^^^^^^^^^^^^^^^^^ .. code:: shell pgd node node-2 setup --dsn "host=host-2 port=5432 user=pgdadmin dbname=pgddb" \ - -listen-addr "localhost,host-2" \ - D /var/lib/edb-pge/17/main - -cluster-dsn "host=host-1 port=5432 user=pgdadmin dbname=pgddb" This command is similar to the first node, but we are setting up ``node-2`` . The ``--dsn`` option is the connection string for the node, which is typically set to ``host=hostname port=5432 user=pgdadmin dbname=pgd`` . The ``cluster-dsn`` must point to an active node, it can point to connection manager, or proxy endpoint etc., CLI will get the real DSN of the node behind it. In this case, we are setting it to ``host=host-1 port=5432 user=pgdadmin dbname=pgd`` , which is the connection string for the first node in the cluster. Configuring a third node ^^^^^^^^^^^^^^^^^^^^^^^^ .. code:: shell pgd node node-3 setup --dsn "host=host-3 port=5432 user=pgdadmin dbname=pgddb" \ - -listen-addr "localhost,host-3" \ - -cluster-dsn "host=host-1 port=5432 user=pgdadmin dbname=pgddb" \ - D /var/lib/edb-pge/17/main This command is similar to the second node, but we are setting up ``node-3`` . The ``--dsn`` option is the connection string for the node, which is typically set to ``host=hostname port=5432 user=pgdadmin dbname=pgd`` . The ``cluster-dsn`` must point to an active node, it can point to connection manager, or proxy endpoint etc., CLI will get the real DSN of the node behind it. In this case, we are setting it to ``host=host-1 port=5432 user=pgdadmin dbname=pgd`` , which is the connection string for the first node in the cluster. Joining a parted and dropped node to the cluster ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. code:: shell pgd node node-2 setup --dsn "host=host-2 port=5432 user=pgdadmin dbname=pgddb" \ - -listen-addr "localhost,host-2" \ - -cluster-dsn "host=host-1 port=5432 user=pgdadmin dbname=pgddb" \ - D /var/lib/edb-pge/17/main This command is similar to the setting up the subsequent nodes, but we are setting up ``node-2`` again. The ``--dsn`` option is the connection string for the node, which is typically set to ``host=hostname port=5432 user=pgdadmin dbname=pgd`` . The ``cluster-dsn`` must point to an active node, it can point to connection manager, or proxy endpoint etc., CLI will get the real DSN of the node behind it. In this case, we are setting it to ``host=host-1 port=5432 user=pgdadmin dbname=pgd`` , which is the connection string for the first node in the cluster. This is useful when a node has been ``parted`` and ``dropped`` from the cluster for some activity like maintenance and needs to be rejoined to the cluster. The command will perform a logical join of the node to the cluster, which will create a new node in the cluster and start streaming replication from the remote node. Configuring the first node with TDE options ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. code:: shell pgd node node-1 setup \ - -dsn "host=host-1 port=5432 user=pgdadmin dbname=pgddb" \ - -listen-addr "localhost,host-1" \ - -group-name group-1 --cluster-name pgd \ - D /var/lib/edb-pge/17/main \ - -data-encryption \ - -key-wrap-command "openssl enc -e -aes-128-cbc -pbkdf2 -pass pass:secret -out %p" \ - -key-unwrap-command "openssl enc -d -aes-128-cbc -pbkdf2 -pass pass:secret -in %p" The ``pgd node setup`` command for a TDE-enabled cluster is similar to the standard command, but requires three additional TDE-specific options: ``--data-encryption`` , ``--key-wrap-command`` , and ``--key-unwrap-command`` . For simplicity, the values for ``--key-wrap-command`` and ``--key-unwrap-command`` in this example use a simple passphrase. However, you can configure more secure mechanisms, such as an external Key Management Service (KMS). The example doesn’t explicitly set the ``--data-encryption-keylen`` option, defaulting the key length to 128 bits. Configuring a second node with TDE options ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. code:: shell pgd node node-2 setup \ - -dsn "host=host-2 port=5432 user=pgdadmin dbname=pgddb" \ - -listen-addr "localhost,host-2" \ - D /var/lib/edb-pge/17/main \ - -cluster-dsn "host=host-1 port=5432 user=pgdadmin dbname=pgddb" \ - -key-wrap-command "openssl enc -e -aes-128-cbc -pbkdf2 -pass pass:secret -out %p" \ - -key-unwrap-command "openssl enc -d -aes-128-cbc -pbkdf2 -pass pass:secret -in %p" This command is similar to the setup used for the first node, retaining the required TDE options. The remaining arguments are consistent with the standard command used when setting up subsequent nodes in a cluster. The values of the TDE options must be identical to those provided for the initial node.