The target-postgres Meltano loader sends data into PostgreSQL after it was pulled from a source using an extractor.

Alternative variants #

Multiple variants of target-postgres are available. This document describes the meltano variant.

Alternative variants are:

Getting Started #

Prerequisites #

If you haven't already, follow the initial steps of the Getting Started guide:

  1. Install Meltano
  2. Create your Meltano project
  3. Add an extractor to pull data from a source

Installation and configuration #

  1. Add the target-postgres loader to your project using meltano add :

    meltano add loader target-postgres --variant meltano
  2. Configure the settings below using meltano config .

Next steps #

Follow the remaining steps of the Getting Started guide:

  1. Run a data integration (EL) pipeline
If you run into any issues, learn how to get help.

Capabilities #

These capabilities can also be overriden by specifying the capabilities key in your meltano.yml file.

Settings #

target-postgres requires the configuration of one of the following groups of settings:

The settings for loader target-postgres that are known to Meltano are documented below. To quickly find the setting you're looking for, use the Table of Contents at the top of the page.

You can override these settings or specify additional ones in your meltano.yml by adding the settings key. Please consider adding any settings you have defined locally to this definition on MeltanoHub by making a pull request to the YAML file that defines the settings for this loader.

User (user) #

How to use #

Manage this setting using meltano config or an environment variable:

meltano config target-postgres set user <user>

export TARGET_POSTGRES_USER=<user>

Password (password) #

How to use #

Manage this setting using meltano config or an environment variable:

meltano config target-postgres set password <password>

export TARGET_POSTGRES_PASSWORD=<password>

Host (host) #

How to use #

Manage this setting using meltano config or an environment variable:

meltano config target-postgres set host <host>

export TARGET_POSTGRES_HOST=<host>

Port (port) #

How to use #

Manage this setting using meltano config or an environment variable:

meltano config target-postgres set port 5502

export TARGET_POSTGRES_PORT=5502

Database Name (dbname) #

How to use #

Manage this setting using meltano config or an environment variable:

meltano config target-postgres set dbname <dbname>

export TARGET_POSTGRES_DBNAME=<dbname>

URL (url) #

Lets you set user, password, host, port, and dbname in one go using a postgresql:// URI.

Takes precedence over the other settings when set.

How to use #

Manage this setting using meltano config or an environment variable:

meltano config target-postgres set url <url>

export TARGET_POSTGRES_URL=<url>

Schema (schema) #

Note that $MELTANO_EXTRACT__LOAD_SCHEMA will expand to the value of the load_schema extra for the extractor used in the pipeline, which defaults to the extractor’s namespace, e.g. tap_gitlab for tap-gitlab.

Name of the schema where the tables will be created. If schema_mapping is not defined then every stream sent by the tap is loaded into this schema.

How to use #

Manage this setting using meltano config or an environment variable:

meltano config target-postgres set schema <schema>

export TARGET_POSTGRES_SCHEMA=<schema>

Troubleshooting #

ld, clang, lssl, or linker Errors #

If you have an error message like:

  • ld: library not found for -lssl
  • clang: error: linker command failed with exit code 1
  • error: command 'clang' failed with exit status 1

These error messages indicates that there is a problem installing OpenSSL.

This Stack Overflow answer has specific recommendations on setting the LDFLAGS and/or CPPFLAGS environment variables. Set those prior to running meltano add loader target-postgres.

Looking for help? #

If you're having trouble getting the target-postgres loader to work, look for an existing issue in its repository, file a new issue, or join the Meltano Slack community and ask for help in the #plugins-general channel.

Found an issue on this page? #

This page is generated from a YAML file that you can contribute changes to. Edit it on GitHub!