SFTP
Table of Contents
The tap-sftp
Meltano extractor pulls data from SFTP that can then be sent to a destination using a loader.
-
- Repository: https://github.com/singer-io/tap-sftp
-
-
-
-
-
-
- Maintainer: Stitch
-
-
Alternative variants #
Multiple
variants
of tap-sftp
are available.
This document describes the default singer-io
variant,
which is recommended for new users.
Alternative variants are:
Getting Started #
Prerequisites #
If you haven't already, follow the initial steps of the Getting Started guide:
Installation and configuration #
-
Add the
tap-sftp
extractor to your project usingmeltano add
:meltano add extractor tap-sftp
-
Configure the settings below using
meltano config
.
Next steps #
Follow the remaining steps of the Getting Started guide:
- Select entities and attributes to extract
- Add a loader to send data to a destination
- Run a data integration (EL) pipeline
Capabilities #
Settings #
tap-sftp
requires the
configuration
of the following settings:
The settings for extractor tap-sftp
that are known to Meltano are documented below.
To quickly find the
setting you're looking for, use the Table of Contents at
the top of the page.
Host (host
)
#
-
Environment variable:
TAP_SFTP_HOST
The SFTP server hostname.
How to use #
Manage this setting using
meltano config
or an
environment variable:
meltano config tap-sftp set host <host>
export TAP_SFTP_HOST=<host>
Port (port
)
#
-
Environment variable:
TAP_SFTP_PORT
The SFTP server port.
How to use #
Manage this setting using
meltano config
or an
environment variable:
meltano config tap-sftp set port 1234
export TAP_SFTP_PORT=1234
Username (username
)
#
-
Environment variable:
TAP_SFTP_USERNAME
The username for connecting the the SFTP server.
How to use #
Manage this setting using
meltano config
or an
environment variable:
meltano config tap-sftp set username <username>
export TAP_SFTP_USERNAME=<username>
Password (password
)
#
-
Environment variable:
TAP_SFTP_PASSWORD
The pass for connecting the the SFTP server.
How to use #
Manage this setting using
meltano config
or an
environment variable:
meltano config tap-sftp set password <password>
export TAP_SFTP_PASSWORD=<password>
Private Key File (private_key_file
)
#
-
Environment variable:
TAP_SFTP_PRIVATE_KEY_FILE
A path to the private key file for connecting the the SFTP server.
How to use #
Manage this setting using
meltano config
or an
environment variable:
meltano config tap-sftp set private_key_file <private_key_file>
export TAP_SFTP_PRIVATE_KEY_FILE=<private_key_file>
Tables (tables
)
#
-
Environment variable:
TAP_SFTP_TABLES
An array that consists of one or more objects that describe how to find files and emit records. Required in each table object - table_name
, search_pattern
,search_prefix
, key_properties
, delimiter
.
How to use #
Manage this setting using
meltano config
or an
environment variable:
meltano config tap-sftp set tables '[...]'
export TAP_SFTP_TABLES='[...]'
Start Date (start_date
)
#
-
Environment variable:
TAP_SFTP_START_DATE
Determines how much historical data will be extracted. Please be aware that the larger the time period and amount of data, the longer the initial extraction can be expected to take.
How to use #
Manage this setting using
meltano config
or an
environment variable:
meltano config tap-sftp set start_date YYYY-MM-DDTHH:MM:SSZ
export TAP_SFTP_START_DATE=YYYY-MM-DDTHH:MM:SSZ
Request Timeout (request_timeout
)
#
-
Environment variable:
TAP_SFTP_REQUEST_TIMEOUT
The SFTP connection timeout in seconds. Defaults to 300.
How to use #
Manage this setting using
meltano config
or an
environment variable:
meltano config tap-sftp set request_timeout 1234
export TAP_SFTP_REQUEST_TIMEOUT=1234
Looking for help? #
If you're having trouble getting the
tap-sftp
extractor to work, look for an
existing issue in its repository, file a new issue,
or
join the Meltano Slack community
and ask for help in the #plugins-general
channel.
Found an issue on this page? #
This page is generated from a YAML file that you can contribute changes to. Edit it on GitHub!