The tap-sftp Meltano extractor pulls data from SFTP that can then be sent to a destination using a loader.

Alternative variants #

Multiple variants of tap-sftp are available. This document describes the default singer-io variant, which is recommended for new users.

Alternative variants are:

Getting Started #

Prerequisites #

If you haven't already, follow the initial steps of the Getting Started guide:

  1. Install Meltano
  2. Create your Meltano project

Installation and configuration #

  1. Add the tap-sftp extractor to your project using meltano add : 

    meltano add extractor tap-sftp
  2. Configure the settings below using meltano config .

Next steps #

Follow the remaining steps of the Getting Started guide:

  1. Select entities and attributes to extract
  2. Add a loader to send data to a destination
  3. Run a data integration (EL) pipeline
If you run into any issues, learn how to get help.

Capabilities #

Settings #

tap-sftp requires the configuration of the following settings:

The settings for extractor tap-sftp that are known to Meltano are documented below. To quickly find the setting you're looking for, use the Table of Contents at the top of the page.

Host (host) #

The SFTP server hostname.

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config tap-sftp set host <host>

export TAP_SFTP_HOST=<host>

Port (port) #

The SFTP server port.

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config tap-sftp set port 1234

export TAP_SFTP_PORT=1234

Username (username) #

The username for connecting the the SFTP server.

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config tap-sftp set username <username>

export TAP_SFTP_USERNAME=<username>

Password (password) #

The pass for connecting the the SFTP server.

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config tap-sftp set password <password>

export TAP_SFTP_PASSWORD=<password>

Private Key File (private_key_file) #

A path to the private key file for connecting the the SFTP server.

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config tap-sftp set private_key_file <private_key_file>

export TAP_SFTP_PRIVATE_KEY_FILE=<private_key_file>

Tables (tables) #

An array that consists of one or more objects that describe how to find files and emit records. Required in each table object - table_name, search_pattern,search_prefix, key_properties, delimiter.

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config tap-sftp set tables '[...]'

export TAP_SFTP_TABLES='[...]'

Start Date (start_date) #

Determines how much historical data will be extracted. Please be aware that the larger the time period and amount of data, the longer the initial extraction can be expected to take.

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config tap-sftp set start_date YYYY-MM-DDTHH:MM:SSZ

export TAP_SFTP_START_DATE=YYYY-MM-DDTHH:MM:SSZ

Request Timeout (request_timeout) #

The SFTP connection timeout in seconds. Defaults to 300.

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config tap-sftp set request_timeout 1234

export TAP_SFTP_REQUEST_TIMEOUT=1234

Looking for help? #

If you're having trouble getting the tap-sftp extractor to work, look for an existing issue in its repository, file a new issue, or join the Meltano Slack community and ask for help in the #plugins-general channel.

Found an issue on this page? #

This page is generated from a YAML file that you can contribute changes to. Edit it on GitHub!