Hadoop Foreign Data Wrapper Guide

The Hadoop Foreign Data Wrapper (hdfs_fdw) is a Postgres extension that allows you to access data that resides on a Hadoop file system from EDB Postgres Advanced Server. The foreign data wrapper makes the Hadoop file system a read-only data source that you can use with Postgres functions and utilities, or in conjunction with other data that resides on a Postgres host.

The Hadoop Foreign Data Wrapper can be installed with an RPM package. You can download an installer from the EDB website.

This guide uses the term Postgres to refer to an instance of EDB Postgres Advanced Server.

• What’s New
• Requirements Overview
  • Supported Versions
  • Supported Platforms
• Architecture Overview
• Supported Authentication Methods
  • Using LDAP Authentication
  • Using NOSASL Authentication
• Installing the Hadoop Foreign Data Wrapper
  • Installing the Hadoop Foreign Data Wrapper using an RPM Package
    • On RHEL 7
    • On RHEL 8
    • On CentOS 7
    • On CentOS 8
  • Installing the Hadoop Foreign Data Wrapper on a Debian or Ubuntu Host
• Features of the Hadoop Foreign Data Wrapper
  • Automated Cleanup
• Configuring the Hadoop Foreign Data Wrapper
  • CREATE EXTENSION
  • CREATE SERVER
  • CREATE USER MAPPING
  • CREATE FOREIGN TABLE
    • Data Type Mappings
• Using the Hadoop Foreign Data Wrapper
  • Using HDFS FDW with Apache Hive on Top of Hadoop
  • Using HDFS FDW with Apache Spark on Top of Hadoop
• Identifying the Hadoop Foreign Data Wrapper Version
• Conclusion