Installing Hybrid Manager on AWS EKS

Installing Hybrid Manager on AWS EKS#

You can install Hybrid Manager (HM) on Amazon EKS using Helm and the bootstrap kit.

Important

Completed the Prerequisites for AI Factory on Hybrid Manager before you start this part of the installation process.

Set the version to install#

The version of the EDB Software Deployment platform is set by the environment variable EDB_PLATFORM_VERSION. This should be set to the version of the platform you wish to install. For example, to install version v1.2.0, you would set the environment variable as follows:

export EDB_PLATFORM_VERSION="v1.2.0"

Set your token#

This installation process requires that you save your EDB subscription token as an environment variable. You can obtain it by going to your EDB Account Profile . (Log in if prompted to.) There you will find an entry for Repos 2.0 token:

Repo Token

Repo Token#

Take this value and set it as an environment variable:

export EDB_SUBSCRIPTION_TOKEN="your-token"

Obtain the bootstrap kit#

Hybrid Manager is installed using a bootstrap kit. This kit is a collection of scripts and Helm charts that are used to install the EDB Software Deployment platform.

Copy all files to a directory on your local machine and cd into that directory.

Install secrets#

The bootstrap process requires a number of secrets to be installed in the Kubernetes cluster. These secrets are used to authenticate with the EDB Download Repository and to store the credentials for the EDB Software Deployment platform.

to install the secrets required for the bootstrap process, run the install-secrets.sh script. (As you already set the token as an environment variable, you can pipe it into the script.)

echo $EDB_SUBSCRIPTION_TOKEN | $SHELL eks-install-secrets.sh

Enter the password for pgai-platform@docker.enterprisedb.com
Creating secret edb-cred
namespace/upm-replicator configured
secret/edb-cred configured
namespace/edbpgai-bootstrap configured
secret/edb-cred configured
secret/edb-cred annotated
PG_CONFOUNDING_KEY is 01234567890123456789ABCDEFG - store safely
Installation completed

This process creates secrets needed for Hybrid Manager to run. You can find out more about these secrets in:

Prepare the values file#

The bootstrap process requires a number of values to be set in the Helm chart. These values are stored in a file called values.yaml . You can create this file by running the prepare.sh script, which uses variables from the default-env.sh file. Edit the default-env.sh file to set the values you require.

This is the default default-env.sh file:

#  exports for EKS

export EDB_PLATFORM_VERSION="v1.2.0"
export EDB_TARGET_PLATFORM="eks"
export CONTAINER_REGISTRY_URI="docker.enterprisedb.com/pgai-platform"
export IMAGESET_REGISTRY_URI=$CONTAINER_REGISTRY_URI
export IMAGESET_AUTHTYPE="token"
export PORTAL_DOMAIN_NAME="portal.foo.network"
export TRANSPORTER_RW_SERVICE_DOMAIN_NAME="transporter.foo.network"
export BEACON_SERVICE_DOMAIN_NAME="beacon.foo.network"
export AUTHENTICATION_EMAIL="owner@mycompany.com"
export AUTHENTICATION_USER="owner@mycompany.com"
export LOCATION_NAME="default-location"

#  Set a password hash for the user or pass a password to have it hashed for you.

#  If you pass a password, you will need to remove it from your history.

#  You can hash a password using the following command:

#  echo -n "password" | htpasswd -BinC 10 admin | cut -d: -f2

#

# export AUTHENTICATION_PASSWORD="password"

export AUTHENTICATION_PASSWORD_HASH="$2y$10$vKOAXfLHbeV1OQxMpxlLdOIwnX.JAN.ZrD9ZU//ocrNQwhQIMtXhy"

Here is a brief guide to the variables in the default-env.sh file.

Variable

Description

EDB_TARGET_PLATFORM

The target platform for the installation. Set to eks as we are installing on Amazon EKS.

CONTAINER_REGISTRY_URI

The URI of the container registry to use. This is set to the EDB Download Repository.

IMAGESET_REGISTRY_URI

The URI of the image set registry. This is set to the EDB Download Repository.

IMAGESET_AUTHTYPE

The authentication type for the image set registry. Set to token as we are using a token to authenticate with the EDB Download Repository.

PORTAL_DOMAIN_NAME

The domain will serve as the public URL for accessing the HM Console. See recommendation below.

TRANSPORTER_RW_SERVICE_DOMAIN_NAME

The domain name for the Data Migration Service (internally transporter). See recommendation below.

BEACON_SERVICE_DOMAIN_NAME

The domain name for the beacon service.

AUTHENTICATION_EMAIL

The email address of the user to create.

AUTHENTICATION_USER

The username of the user to create.

AUTHENTICATION_PASSWORD

The password of the user to create. This can be set as plain text or as a hash with AUTHENTICATION_PASSWORD_HASH.

AUTHENTICATION_PASSWORD_HASH

The hash of the password to create.

LOCATION_NAME

The location name for the Agent, formerly known as Beacon Agent.

!!!note Recommendations for PORTAL_DOMAIN_NAME and TRANSPORTER_RW_SERVICE_DOMAIN_NAME

For consistency and ease of management, derive your TRANSPORTER_RW_SERVICE_DOMAIN_NAME from your PORTAL_DOMAIN_NAME. The most common approach is to replace the portal subdomain with dms .

The following table illustrates this relationship using a variety of examples based on the top-level domain examplecompany.com .

PORTAL_DOMAIN_NAME

TRANSPORTER_RW_SERVICE_DOMAIN_NAME

portal.examplecompany.com

dms.examplecompany.com

portal-dev.examplecompany.com

dms-dev.examplecompany.com

portal.hm001.examplecompany.com

dms.hm001.examplecompany.com

portal-hm-us-east-001.examplecompany.com

dms-hm-us-east-001.examplecompany.com

Run the prepare.sh script to create the values.yaml file.

$SHELL prepare.sh

This create a values.yaml file with the values set in the default-env.sh file.

Install the Helm chart repo with the following command:

helm repo add edbpgai "https://downloads.enterprisedb.com/${EDB_SUBSCRIPTION_TOKEN}/pgai-platform/helm/charts"

Once the charts repository is added, ensure it is up to date:

helm repo update

Now you can create the values.yaml file from the helm repo using the following command:

helm show values edbpgai/edbpgai-bootstrap > values.yaml

You now have a values.yaml file with the chart’s default values for the bootstrap process.

You need to create two values for the bootstrap process. These are the AES_256_KEY and the AUTHENTICATION_PASSWORD_HASH. You can create these values with the following commands:

export AES_256_KEY=$(openssl rand -base64 32)

If you have a password to hash, you can do so with the following command:

export AUTHENTICATION_PASSWORD_HASH=$(echo -n $AUTHENTICATION_PASSWORD | htpasswd -BinC 10 admin | cut -d: -f2)

You now need to edit values.yaml file and set the values required for your installation. Find the appropriate key in the values.yaml file and set the values as follows. Replace $<name> entries with the value you have set the environment variables of the same name to.

  1. Sets bootstrapImageName to $CONTAINER_REGISTRY_URI followed by “/edbpgai-bootstrap/bootstrap-” and then $EDB_TARGET_PLATFORM. eg, given the values above, this would be docker.enterprisedb.com/pgai-platform/edbpgai-bootstrap/bootstrap-eks .

  2. Set bootstrapImageTag to $EDB_PLATFORM_VERSION.

  3. Set parameters.upm-istio.cookie_aeskey to $AES_256_KEY.

  4. Set containerRegistryURL to $CONTAINER_REGISTRY_URI.

  5. Set system to $EDB_TARGET_PLATFORM.

  6. Set parameters.global.portal_domain_name to $PORTAL_DOMAIN_NAME.

  7. Set parameters.transporter-rw-service.domain_name to $TRANSPORTER_RW_SERVICE_DOMAIN_NAME.

  8. Set parameters.transporter-dp-agent.rw_service_url to https:// followed by $TRANSPORTER_RW_SERVICE_DOMAIN_NAME followed by /transporter . Following the first of the previously provided examples, this would be https://dms.examplecompany.com/transporter .

  9. Set parameters.upm-beacon.server_host to $BEACON_SERVICE_DOMAIN_NAME. 10. Set pgai.portal.authentication.staticPasswords[0].email to $AUTHENTICATION_EMAIL. 11. Set pgai.portal.authentication.staticPasswords[0].hash to $AUTHENTICATION_PASSWORD_HASH. 12. Set pgai.portal.authentication.staticPasswords[0].username to $AUTHENTICATION_USER. 13. Set pgai.portal.authentication.staticPasswords[0].userID “c5998173-a605-449a-a9a5-4a9c33e26df”. 14. Set beaconAgent.location to $LOCATION_NAME

Apply the Helm chart#

With the values.yaml file prepared, you can apply the Helm chart to start the bootstrap process. This create the necessary resources in the Kubernetes cluster to start the bootstrap process.

helm upgrade -n edbpgai-bootstrap \
   --install -f ./values.yaml \
   --version "${EDB_PLATFORM_VERSION/-appl/+appl}" \
   edbpgai-bootstrap edbpgai/edbpgai-bootstrap

This command installs the bootstrap container in the Kubernetes cluster. The bootstrap container then installs the rest of the EDB Software Deployment platform.

Follow the bootstrap process#

The bootstrap process takes time to complete. You can follow the progress by looking at the logs of the bootstrap container:

kubectl logs -f -l app=edbpgai-bootstrap -n edbpgai-bootstrap

Configure host name resolution#

Obtain the IP address of your ingress gateway. Run:

kubectl -n istio-system get service istio-ingressgateway -o jsonpath={.status.loadBalancer.ingress[0].hostname} | strings | nslookup

This command returns output like:

Server:         192.168.111.1
Address:        192.168.111.1#53

Non-authoritative answer:
Name:   k8s-istiosys-istioing-b3dc9b4f3c-81dc5c45b1d5f6d2.elb.us-east-1.amazonaws.com
Address: 52.70.180.102

You now need to configure your DNS service or your /etc/hosts file to point the domains specified in default-env.sh to the IP address you obtained from the command above. Those domains are in PORTAL_DOMAIN_NAME , TRANSPORTER_RW_SERVICE_DOMAIN_NAME , and BEACON_SERVICE_DOMAIN_NAME . In the example above, they are set to portal.examplecompany.com , dms.examplecompany.com , and beacon.examplecompany.com respectively.

Configure DNS#

In your DNS configuration, create a CNAME record for each of these domains that points to the IP address you obtained. So for our example, you would create the following records in your DNS zone file for examplecompany.com :

cluster IN A 52.70.180.102
portal      IN CNAME cluster.examplecompany.com
transporter IN CNAME cluster.examplecompany.com
beacon      IN CNAME cluster.examplecompany.com

This is a simplified example. The actual configuration will depend on your DNS provider and how you manage your DNS records.

Configure /etc/hosts (if no DNS service)#

If you don’t have a DNS service configured, consider adding the IP address to your local /etc/hosts file. This addition allows you to access Hybrid Manager Console using the domain name you set in the default-env.sh file.

You can then add the IP address to your /etc/hosts file like this:

echo "52.70.180.102 portal.examplecompany.com dms.examplecompany.com beacon.examplecompany.com" | sudo tee -a /etc/hosts

Substitute the IP address with the value you obtained earlier. This is a temporary solution and should not be used in production environments. We recommend that you use a proper DNS service for production deployments. The settings in the /etc/hosts file work only on the machine where you set them.

Next steps#

You can now verify the installation by connecting to the HM Console .

default-env.sh#

# exports for EKS
export EDB_PLATFORM_VERSION="v1.2.0"
export EDB_TARGET_PLATFORM="eks"
export CONTAINER_REGISTRY_URI="docker.enterprisedb.com/pgai-platform"
export IMAGESET_REGISTRY_URI=$CONTAINER_REGISTRY_URI
export IMAGESET_AUTHTYPE="token"
export PORTAL_DOMAIN_NAME="portal.foo.network"
export TRANSPORTER_RW_SERVICE_DOMAIN_NAME="transporter.foo.network"
export BEACON_SERVICE_DOMAIN_NAME="beacon.foo.network"
export AUTHENTICATION_EMAIL="owner@mycompany.com"
export AUTHENTICATION_USER="owner@mycompany.com"
export LOCATION_NAME="default-location"

# Set a password hash for the user or pass a password to have it hashed for you.
# If you pass a password, you will need to remove it from your history.
# You can hash a password using the following command:
# echo -n "password" | htpasswd -BinC 10 admin | cut -d: -f2
#

# export AUTHENTICATION_PASSWORD="password"
# If setting AUTHENTICATION_PASSWORD_HASH, ensure it is single quoted to prevent variable expansion (e.g. $2y$10$...).
export AUTHENTICATION_PASSWORD_HASH=$2y$10$vKOAXfLHbeV1OQxMpxlLdOIwnX.JAN.ZrD9ZU//ocrNQwhQIMtXhy

eks-install-secrets.sh#

registry="docker.enterprisedb.com"
username="pgai-platform"
password=""

echo "Enter the password for ${username}@${registry}"
read -s password

if [ -z "$password" ]; then
    echo 1>&2 "Error: password is required" >&2
    echo 1>&2
    usage
    exit 1
fi

kubectl create ns edbpgai-bootstrap
kubectl create ns upm-replicator

kubectl create secret docker-registry edb-cred \
    -n edbpgai-bootstrap \
    --docker-server="${registry}" \
    --docker-username="${username}" \
    --docker-password="${password}"

kubectl create secret docker-registry edb-cred \
    -n upm-replicator \
    --docker-server="${registry}" \
    --docker-username="${username}" \
    --docker-password="${password}"

kubectl annotate secret -n upm-replicator edb-cred replicator.v1.mittwald.de/replicate-to="*" --overwrite

kubectl create namespace upm-griptape

FERNET_KEY=$(dd if=/dev/urandom bs=32 count=1 2>/dev/null | base64)

kubectl apply -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
    name: fernet-secret
    namespace: upm-griptape
stringData:
    FERNET_KEY: ${FERNET_KEY}
EOF

kubectl create namespace upm-lakekeeper

PG_CONFOUNDING_KEY=$(dd if=/dev/urandom bs=32 count=1 2>/dev/null | base64)

kubectl apply -f - <<EOF
  apiVersion: v1
  kind: Secret
  metadata:
    name: pg-confounding-key
    namespace: upm-lakekeeper
  stringData:
    PG_CONFOUNDING_KEY: ${PG_CONFOUNDING_KEY}
EOF

echo "PG_CONFOUNDING_KEY is ${PG_CONFOUNDING_KEY} - store safely"

prepare.sh#

# !/bin/bash
echo "HM Helm values.yaml preparation\n"

#  If EDB_PLATFORM_VERSION is not set, error and exit

if [ -z "$EDB_PLATFORM_VERSION" ]; then
  echo "EDB_PLATFORM_VERSION is not set, please set it to the version of EDB Platform you are deploying"
  exit
fi

#  Test for the presence of yq

if ! command -v yq &> /dev/null
then
    echo "yq could not be found, please install it"
    exit
fi

#  Import the eks-default-env file

source ./default-env.sh

#  Test for presence of helm repo

value=`helm repo list -o yaml | yq eval contains([{ "name":"edbpgai"}]) -`
if [[ "$value" == "false" ]]; then
  echo "EDB Platform Helm repository not found, adding in now"
  helm repo add edbpgai "https://downloads.enterprisedb.com/${EDB_SUBSCRIPTION_TOKEN}/pgai-platform/helm/charts/"
fi

#  Always update the helm repo

helm repo update
helm show values edbpgai/edbpgai-bootstrap --version $EDB_PLATFORM_VERSION > values.yaml
echo "\n\n"

#  Prepare version dependent values

export AES_256_KEY=$(openssl rand -base64 32)

if [[ "$EDB_PLATFORM_VERSION" == v1.2.* ]]; then
    echo "EDB Platform version is v1.2.*"
    yq -i .bootstrapImageName = env(CONTAINER_REGISTRY_URI)+"/edbpgai-bootstrap/bootstrap-"+env(EDB_TARGET_PLATFORM) |
    .bootstrapImageTag=env(EDB_PLATFORM_VERSION) |
    del(.bootstrapImage) |
    .parameters.upm-istio-gateway.cookie_aeskey = env(AES_256_KEY) |
    del(.parameters.upm-beacon-ff-base) |
    .beaconAgent.provisioning.imagesetDiscoveryContainerRegistryURL= env(IMAGESET_REGISTRY_URI) |
    .beaconAgent.provisioning.imagesetDiscoveryContainerRegistryAuthType= env(IMAGESET_AUTHTYPE) |
    .containerRegistryURL = env(CONTAINER_REGISTRY_URI) values.yaml
fi

#  Prepare values.yaml for EKS

#  if AUTHENTICATION_PASSWORD_HASH is not set, generate it

if [ -z "$AUTHENTICATION_PASSWORD_HASH" ]; then
  echo "Generating password hash - please ensure you remove the password from your history"
  export AUTHENTICATION_PASSWORD_HASH=$(echo -n $AUTHENTICATION_PASSWORD | htpasswd -BinC 10 admin | cut -d: -f2)
else
  echo "Using prehashed password"
fi

yq -i .system = env(EDB_TARGET_PLATFORM) |
.parameters.global.portal_domain_name = env(PORTAL_DOMAIN_NAME) |
.parameters.transporter-rw-service.domain_name = env(TRANSPORTER_RW_SERVICE_DOMAIN_NAME) |
.parameters.transporter-dp-agent.rw_service_url= "https://"+env(TRANSPORTER_RW_SERVICE_DOMAIN_NAME)+"/transporter" |
.parameters.upm-beacon.server_host= env(BEACON_SERVICE_DOMAIN_NAME) |
.pgai.portal.authentication.staticPasswords[0].email=env(AUTHENTICATION_EMAIL) |
.pgai.portal.authentication.staticPasswords[0].hash=env(AUTHENTICATION_PASSWORD_HASH) |
.pgai.portal.authentication.staticPasswords[0].username=env(AUTHENTICATION_USER) |
.pgai.portal.authentication.staticPasswords[0].userID="c5998173-a605-449a-a9a5-4a9c33e26df" |
.beaconAgent.location=env(LOCATION_NAME) values.yaml

echo "Your values.yaml file is ready"
Contents