Tiered Tables in Hybrid Manager

Tiered Tables in Hybrid Manager#

Tiered Tables in Hybrid Manager (HM) provide an automated solution for managing large time-series or historical datasets by moving older data from PGD clusters to cost-effective object storage in Apache Iceberg format.

Hybrid Manager integrates this capability across:

  • PGD clusters with BDR AutoPartition

  • PGAA extensions for offloading

  • Lakehouse Clusters for querying

  • Centralized catalog services (Lakekeeper or external Iceberg catalogs)

For a conceptual overview of Tiered Tables, see Understanding Tiered Tables with EDB Postgres .

Why use Tiered Tables with Hybrid Manager#

  • Lower storage costs: Offload “cold” data to object storage (Iceberg) and shrink primary PGD transactional tables.

  • Faster transactional performance: Keep “hot” data partitions small for efficient PGD operations.

  • Automated lifecycle management: Move data across tiers automatically based on age.

  • Transparent analytics: Query both hot and cold data via PGD parent table or Lakehouse Cluster.

  • Unified management: Configure and monitor all components through Hybrid Manager.

Key terms and architecture overview#

For definitions of core analytics terms used in Hybrid Manager—such as PGFS, PGAA, Analytics Offload, and Lakehouse Cluster—see Analytics Concepts in Hybrid Manager .

When should I use Tiered Tables in Hybrid Manager?#

Use Tiered Tables in Hybrid Manager when you want to:

  • Manage large time-series datasets in a cost-efficient way.

  • Keep PGD operational tables lean for better performance.

  • Meet compliance needs by keeping older data available but outside of PGD storage.

  • Enable BI users to run historical trend queries without impacting production databases.

  • Automate your data lifecycle with minimal manual intervention.

Use cases for Tiered Tables#

  • Time-series data: Logging, IoT sensor readings, application telemetry.

  • Archival: Long-term retention of cold data for compliance.

  • Historical trend analysis: BI tools querying years of data without impacting PGD performance.

  • Large, append-mostly tables: Keep transactional footprint small while retaining full analytical access.

How Tiered Tables work in your HM architecture#

  • PGD clusters: Manage partitioning and automatic offload of old partitions to Iceberg.

  • PGFS storage locations: Define object storage targets for offload.

  • Iceberg catalogs: Optionally manage offloaded tables in a catalog (Lakekeeper or external).

  • Lakehouse Clusters: Provide scalable analytical compute to query offloaded Iceberg data.

  • Monitoring: Use HM monitoring tools and observability queries to track offload status and storage savings.

Prerequisites within EDB Hybrid Manager#

Before implementing Tiered Tables in HM:

  • Active Hybrid Manager instance

  • Provisioned PGD cluster: Version 6.0+ with PGAA and PGFS extensions enabled

  • Lakehouse Cluster (recommended): For querying offloaded data

  • Catalog service: Optional, but recommended — HM-managed Lakekeeper or external REST-compatible catalog

  • Machine user for catalog (if using catalog): With appropriate catalog data writer/reader permissions

  • Object storage: S3-compatible, with credentials if private

  • User permissions: Database user must have create/alter/execute privileges for BDR and PGAA functions

Main capabilities#

  • Automated partitioning: Define BDR AutoPartition strategy and analytics_offload_period .

  • Storage tiering: Use PGFS or Iceberg catalog targets for offloaded data.

  • Query transparently: PGD parent table queries can hit both local and Iceberg tiers. Lakehouse Clusters can query Iceberg tables directly.

  • Monitor status: Track offload progress, validate Iceberg content, and observe space savings.

Getting started with Tiered Tables in Hybrid Manager#

To begin using Tiered Tables with Hybrid Manager:

  1. Configure PGFS storage for Tiered Tables .

  2. Configure PGD node group for analytics offload .

  3. Configure BDR AutoPartition with analytics offload .

  4. (Optional) Query Tiered Tables from PGD and Lakehouse .

  5. Monitor Tiered Tables status and storage savings .

Observability tips#

  • Use HM dashboards for PGD cluster health and offload progress.

  • Run analytics queries on bdr.analytics_table and partition views.

  • Use pg_total_relation_size() to observe space reclaimed on PGD nodes.

  • Use cloud storage console or analytics to track Iceberg object size growth.

Next topic#

EDB Postgres Lakehouse Clusters on Hybrid Manager

Contents