Hadoop Foreign Data Wrapper Guide

The Hadoop Foreign Data Wrapper (hdfs_fdw) is a Postgres extension that allows you to access data that resides on a Hadoop file system from an Advanced Server or PostgreSQL server. The foreign data wrapper makes the Hadoop file system a read-only data source that you can use with Postgres functions and utilities, or in conjunction with other data that resides on a Postgres host.

The Hadoop Foreign Data Wrapper can be installed with an RPM package. You can download an installer from the EDB website.

This guide uses the term Postgres to refer to an instance of either EDB Postgres Advanced Server or PostgreSQL.

• Requirements Overview
  • Supported Versions
  • Supported Platforms
• Architecture Overview
• Supported Authentication Methods
  • Using LDAP Authentication
  • Using NOSASL Authentication
• Installing the Hadoop Foreign Data Wrapper
  • Installing the Hadoop Foreign Data Wrapper using an RPM Package
  • Installing the Hadoop Foreign Data Wrapper on a Debian or Ubuntu Host
• Configuring the Hadoop Foreign Data Wrapper
  • CREATE EXTENSION
  • CREATE SERVER
  • CREATE USER MAPPING
  • CREATE FOREIGN TABLE
    • Data Type Mappings
• Using the Hadoop Foreign Data Wrapper
  • Using HDFS FDW with Apache Hive on Top of Hadoop
  • Using HDFS FDW with Apache Spark on Top of Hadoop
• Identifying the Hadoop Foreign Data Wrapper Version
• Conclusion