Minimum recommended specifications for the Hybrid Manager

General recommended resource requirements

When configuring a Kubernetes cluster to run PGAIHM, the minimum recommended general resource requirements are:

New Kubernetes cluster

Running PGAIHM on an existing Kubernetes cluster is not supported at this time. Therefore, you must create a new Kubernetes cluster for the purposes of hosing PGAIHM.

Compute

When considering compute requirements for the Hybrid Manager (HM), it is important to understand that PGAIHM itself is relatively lightweight out of the box and it’s only the workloads from different use cases which may require scaling.

The minimum requirements for PGAIHM are 16GB of RAM, 4 vCPUs, and 50GB of disk on one machine; these are the resources PGAIHM consumes upon installation. This is only recommended for testing PGAIHM, not for production use.

Handling workloads

For general production use, 1-20 Postgres clusters (depending on size of the workload), the following compute recommendations are suggested:

• 6 Kubernetes worker nodes, 3 for HM control components, 3 for HM database nodes, each with:

• 8+ cores

• 32 GB of memory

• 64 GB of storage

For larger data workloads, more worker nodes are likely required.

AI

If planning to run AI workloads, add at least:

• 1 GPU node, 16-32 cores, 30 Gi storage space

However, if your AI workloads are very large or you have many of them, you likely need more GPU resources. To see exact specification you may need for your particular model needs, consult the NVIDIA documentation .

Object store

• S3 compatible object store

• `Ceph <https://github.com/ceph>`_  ,  `MinIO <https://github.com/minio/minio>`_  ,
  

• `AWS S3 <https://aws.amazon.com/s3/>`_  for EKS deployments
  

Storage Layer

• A Container Storage Interface (CSI) driver compatible with Kubernetes for dynamic provisioning of persistent volumes

• OpenShift Container Storage (OCS) or Rook-Ceph for on-premises deployments

• AWS EBS CSI driver for EKS

• Storage classes

• At least one default storage class configured with dynamic volume provisioning

• For on-premises deployments, NVMe is recommended for optimal performance. For example, systems like the Supermicro SYS-221H-TNR are suitable for deployments requiring high-speed storage.

• SAN - Must use a Storage Area Network (SAN) for centralized and high-performance needs of HM. - Ensure SAN supports encryption at rest and integrates with Kubernetes via a CSI driver. - Provide a dedicated and secure network for SAN traffic.

• Snapshot support - Ensure the CSI driver supports volume snapshots if using snapshot backups in HM

Configuration settings

The following configuration settings are recommended for running HM on Kubernetes:

• Kubernetes version 1.20 or later.

Network settings

• Pod network

• Minimum CIDR range for pod IPs depends on the Cloud Service Provider or your on-premises needs. - Example: /22 or more for AWS EKS is advised

• Service network - Minimum CIDR range for Kubernetes services depends on the CSP or your on-premises needs.

• External access - If accessing HM using the internet, these ingress/egress requirements apply

• DNS - Cluster-wide DNS service must be operational.

Security settings

• Access control

- Ensure that Kubernetes RBAC is configured with the principle of least privelege. Users, services, and pods should only have access to the resources they need.

• Authentication

- Use secure authentication mechanisms such as integration with an identity provider (IDP)
    - LDAP, SAML, or OpenID Connect
- Multi-factor Authentication (MFA)
    - Enforce MFA for all administrator accounts accessing the HM.

• Network security

• Cluster network policies - Implement Kubernetes network policies to control traffic flow between pods, namespaces, and external resources.

• Encrypted communication - Use TLS for all communication between HM components and external clients. - Ensure in-cluster service communication is encrypted

• Firewalls and security groups - For on-premises deployments, configure firewalls to restrict access to Kubernetes API servers and control-plane nodes. - For cloud-based deployments, utilize security groups to restrict inbound and outbound traffic to the cluster

• Data security

• Encryption at rest - Ensure persistent volumes used by HM have encryption enabled - Use a storage backend that supports encryption (e.g. AWS S3 for backups in EKS, NVMe with encryption for local storage on-premises)

• Encryption in transit - Enable encryption for all data transfer between HM components and the database clusters they manage.

• Secrets management - Store all sensitive information (e.g. database passwords, API keys) in Kubernetes Secrets. - Use a secure external secrets manager (e.g. HashiCorp Vault, AWS Secrets Manager) when possible.

• Backup security

• Ensure backup data stored in Barman Cloud or snapshots is encrypted.

• Use RBAC policies for backup storage to limit access to authorized personnel and services.

• Regularly test backup restoration processes to ensure they are secure and effective.

• Monitoring and auditing

• Audit logs - Enable and monitor Kubernetes audit logs for administrative actions and unusual activity

• Log aggregation - Use the intergrated Grafan and Loki stack to centralize and analyze logs for security events.

• Alerting - Configure alerts for critical events such as failed authentications, unauthorized access attempts, or resource policy violations.

• Container Security

• Use container images from trusted sources and scan them regularly for vulnerabilities (e.g with tools like Trivy or Clair)

• Apply Kubernetes Pod Security Standards (PSS) to enforce baseline security configurations for pods.

• Restrict container priveleges - Disable privilege escalation. - Set runAsNonRoot to ture in pod security contexts

• Kubernetes cluster security

• Disable anonymous access to the Kubernetes API server.

• Regularly patch and update Kubernetes and all associated components.

• Physical security (for on-premises deployments)

• SAN Security - Secure physical access to SAN hardware, ensuring it is located in a restricted, access-controlled data center. - Implement redundancy and failover configurations for critical SAN components to ensure high availability. - Isolate SAN network traffic to prevent unauthorized access and reduce the risk of interception.

• Compliance

• Ensure HM deployment adheres to organizational or industry-specific security standards (e.g. SOC 2, HIPAA, GDPR).

• Regularly review compliance and update configurations as standards evolve.

Please see capcacity planning document for guidance on resources based on HM functionality or use cases.

2. Kubernetes Platform and Version Requirements

HM is designed to run on various conformant Kubernetes distributions. Supported platforms include:

• Amazon Elastic Kubernetes Service (EKS)

• Google Kubernetes Engine (GKE)

• Red Hat OpenShift (RHOS)

(Note: Specific version compatibility should be checked against the HM release notes.)

3. Kubernetes Nodes and Processor Requirements

3.1. Node Characteristics

• Operating System: Nodes must be Linux-compatible.

• Processor Architecture: HM components primarily target AMD64/x86-64 architecture. While PostgreSQL itself can run on ARM64, mixed-architecture Kubernetes clusters for HM deployments are not standard and may present support complexities.

3.2. Minimum Environmental Layout

A typical minimum production-viable environment consists of:

• Location: Single city.

• Infrastructure: Single physical data center (for on-premises) OR a single region in a Cloud Service Provider (CSP).

• Kubernetes Cluster: One Kubernetes cluster.

• Node Allocation for HM: - A minimum of 3 Kubernetes nodes are required to host the HM control plane components, ensuring high availability. These can be Kubernetes control plane nodes or worker nodes, labeled appropriately.

• Node Allocation for PostgreSQL (Optional but Recommended): - While not a strict HM requirement (PostgreSQL can be co-located if resources and labels permit), dedicating at least 3 additional Kubernetes worker nodes for PostgreSQL instances is recommended for production workloads to ensure resource isolation and performance.

4. External Services Requirements

HM, running within Kubernetes, relies on several external services that must be provisioned and accessible to the Kubernetes cluster:

• Networking:

• Local network infrastructure (on-premises) or a Virtual Private Cloud (VPC) in a CSP.

• Load Balancer: Required for exposing HM and PostgreSQL services. This is typically a managed service in CSPs. For on-premises RHOS, a solution like MetalLB or an enterprise load balancer must be integrated.

• DNS: Reliable DNS resolution for service discovery within and outside the cluster. Control over DNS records for service exposure is necessary.

• Identity Provider (IDP): Optional, for integrating with existing authentication systems .

• Object Storage: An S3-compatible object storage solution is mandatory for backups and other platform features. This is a managed service in CSPs; for on-premises, a compatible solution must be provisioned and accessible to RHOS.

• Key Management System (KMS): Optional, for managing encryption keys.

• Block Storage: Persistent, high-performance block storage is required for PostgreSQL data volumes and potentially other stateful components of HM.

• In CSPs, this is typically a service like AWS EBS, GCP Persistent Disk, or Azure Disk Storage.

• For on-premises deployments, a Storage Area Network (SAN) or equivalent high-performance storage solution integrated with Kubernetes (via CSI drivers) is necessary. A SAN can be considered functionally equivalent to CSP block storage services in this context.

5. Dependent External Services (Connectivity)

The Kubernetes cluster nodes running HM components and EDB Postgres instances require outbound internet access to the following EDB services:

• EDB Container Registry: For pulling container images. - Endpoint: docker.enterprisedb.com

• EDB Entitlements Service: For license validation and service entitlements. - Endpoint: biganimal.com