Kinesis
Table of Contents
The
target-kinesis
Singer target
sends data into
Kinesis
after it was pulled from a source using a
Singer tap.
-
- Repository: https://github.com/ProntoPro/target-kinesis
-
-
-
-
-
-
-
- Maintainer:
Alternative variants
Multiple
variants
of target-kinesis
are available. This document describes the
default prontopro
variant, which is recommended for new
users.
Alternative variants are:
Standalone usage
Install the package using pip:
pip install git+https://github.com/ProntoPro/target-kinesis.git
For additional instructions, refer to the README in the repository.
Usage with Meltano
Install Meltano, create your Meltano project, and add the target to your project as a custom loader:
meltano add --custom loader target-kinesis
Then, configure the loader, add any Singer tap as an extractor to pull data from a source and run a data integration (EL) pipeline.
Capabilities
Settings
Stream Name (stream_name
)
The name of the Kinesis stream to write to.
AWS S3 Access Key ID (aws_access_key_id
)
S3 Access Key Id. If not provided, aws_profile or AWS_ACCESS_KEY_ID environment variable will be used.
AWS S3 Secret Access Key (aws_secret_access_key
)
S3 Secret Access Key. If not provided, aws_profile or AWS_ACCESS_KEY_ID environment variable will be used.
AWS Region (region
)
The AWS region to use i.e. us-east-2.
Is Firehouse (is_firehose
)
(Default False) Whether to user a Firehouse client or not.
Record Chunks (record_chunks
)
(Default - 10) The max number of records per chunk.
Data Chunks (data_chunks
)
(Default - 1000) The max size of data per chunk.
Partition Key (partition_key
)
(Default - id
) The partition key to use when writing to Kinesis.
Looking for help?
If you're having trouble getting
target-kinesis
to work by itself or with
Meltano, look for an
existing issue in its repository, file a new issue,
or
join the Meltano Slack community
and ask for help in the #plugins-general
channel.
Found an issue on this page?
This page is generated from a YAML file that you can contribute changes to! It is also validated against a JSON Schema used for taps and targets.