The target-kinesis Singer target sends data into Kinesis after it was pulled from a source using a Singer tap.

Alternative variants #

Multiple variants of target-kinesis are available. This document describes the prontopro variant.

Alternative variants are:

Standalone usage #

Install the package using pip: 

pip install git+https://github.com/ProntoPro/target-kinesis.git

For additional instructions, refer to the README in the repository.

Usage with Meltano #

Meltano helps you manage your configuration, incremental replication, and scheduled pipelines.

View the Meltano-specific target-kinesis instructions to learn more.

Capabilities #

These capabilities can also be overriden by specifying the capabilities key in your meltano.yml file.

Settings #

target-kinesis requires the configuration of the following settings:

The settings for target target-kinesis that are known to Meltano are documented below. To quickly find the setting you're looking for, use the Table of Contents at the top of the page.

You can override these settings or specify additional ones in your meltano.yml by adding the settings key. Please consider adding any settings you have defined locally to this definition on MeltanoHub by making a pull request to the YAML file that defines the settings for this target.

Stream Name (stream_name) #

The name of the Kinesis stream to write to.

AWS S3 Access Key ID (aws_access_key_id) #

S3 Access Key Id. If not provided, aws_profile or AWS_ACCESS_KEY_ID environment variable will be used.

AWS S3 Secret Access Key (aws_secret_access_key) #

S3 Secret Access Key. If not provided, aws_profile or AWS_ACCESS_KEY_ID environment variable will be used.

AWS Region (region) #

The AWS region to use i.e. us-east-2.

Is Firehouse (is_firehose) #

(Default False) Whether to user a Firehouse client or not.

Record Chunks (record_chunks) #

(Default - 10) The max number of records per chunk.

Data Chunks (data_chunks) #

(Default - 1000) The max size of data per chunk.

Partition Key (partition_key) #

(Default - id) The partition key to use when writing to Kinesis.

Looking for help? #

If you're having trouble getting the target-kinesis target to work, look for an existing issue in its repository, file a new issue, or join the Meltano Slack community and ask for help in the #plugins-general channel.

Found an issue on this page? #

This page is generated from a YAML file that you can contribute changes to. Edit it on GitHub!