The tap-slack extractor pulls data from Slack that can then be sent to a destination using a loader.
Alternate Implementations
Getting Started
Prerequisites
If you haven't already, follow the initial steps of the Getting Started guide:
Creating the Tap-Slack App
In order to access the records in your workspace, you will need to create a new Slack App. Below is an example App Manifest that you can use for your workspace. If you would like to access additional channels, such as direct messages, you will need to provide additional scopes to your Slack app.
_metadata:
major_version: 1
minor_version: 1
display_information:
name: MeltanoLabs Tap-Slack
description: Slack App to support the implementation of Singer.io tap-slack
long_description: This application is used for extracting channel, user, and message data from the Slack workspace via the tap-slack application. Found on GitHub at https://github.com/MeltanoLabs/tap-slack.
features:
bot_user:
display_name: MeltanoLabs Tap-Slack
always_online: false
oauth_config:
redirect_urls:
- https://meltano.com/
scopes:
bot:
- channels:join
- channels:history
- channels:read
- users:read
- users:read.email
settings:
org_deploy_enabled: false
socket_mode_enabled: false
token_rotation_enabled: false
Once you receive an API key you can test that it has proper access to your Slack app using the following curl command.
curl -X GET -H 'Authorization: Bearer <Your Token>' https://slack.com/api/conversations.list?exclude_archived=False
Installation and configuration
-
Add the tap-slack extractor to your
project using
:meltano add -
Configure the tap-slack
settings using
:meltano config -
Test that extractor settings are valid using
:meltano config
meltano add extractor tap-slackmeltano config tap-slack set --interactivemeltano config tap-slack testNext steps
Follow the remaining steps of the Getting Started guide:
If you run into any issues, learn how to get help.
Capabilities
The current capabilities for
tap-slack
may have been automatically set when originally added to the Hub. Please review the
capabilities when using this extractor. If you find they are out of date, please
consider updating them by making a pull request to the YAML file that defines the
capabilities for this extractor.
This plugin has the following capabilities:
- about
- batch
- catalog
- discover
- schema-flattening
- state
- stream-maps
You can
override these capabilities or specify additional ones
in your meltano.yml by adding the capabilities key.
Settings
The
tap-slack settings that are known to Meltano are documented below. To quickly
find the setting you're looking for, click on any setting name from the list:
api_keyauto_join_channelschannel_typesexcluded_channelsinclude_admin_streamsselected_channelsstart_datethread_lookback_days
You can also list these settings using
with the meltano configlist
subcommand:
meltano config tap-slack list
You can
override these settings or specify additional ones
in your meltano.yml by adding the settings key.
Please consider adding any settings you have defined locally to this definition on MeltanoHub by making a pull request to the YAML file that defines the settings for this plugin.
API Key (api_key)
-
Environment variable:
TAP_SLACK_API_KEY
The token to authenticate against the Slack API service
Configure this setting directly using the following Meltano command:
meltano config tap-slack set api_key [value]Auto Join Channels (auto_join_channels)
-
Environment variable:
TAP_SLACK_AUTO_JOIN_CHANNELS -
Default Value:
false
Whether the bot user should attempt to join channels that it has not yet joined. The bot user must be a member of the channel to retrieve messages.
Configure this setting directly using the following Meltano command:
meltano config tap-slack set auto_join_channels [value]Channel Types (channel_types)
-
Environment variable:
TAP_SLACK_CHANNEL_TYPES -
Default Value:
["public_channel"]
The types of conversations the tap will attempt to extract data from. Must be one of 'public_channel', 'mpim', 'private_channel', or 'im'. Note that the Slack app must have the appropriate privileges and be a member of the conversations to sync messages.
Configure this setting directly using the following Meltano command:
meltano config tap-slack set channel_types [value]Excluded Channels (excluded_channels)
-
Environment variable:
TAP_SLACK_EXCLUDED_CHANNELS
A list of channel IDs that should not be retrieved. Excluding overrides a selected setting, so if a channel is included in both selected and excluded, it will be excluded.
Configure this setting directly using the following Meltano command:
meltano config tap-slack set excluded_channels [value]Include Admin Streams (include_admin_streams)
-
Environment variable:
TAP_SLACK_INCLUDE_ADMIN_STREAMS -
Default Value:
false
Whether to include streams that require admin privileges or not. If the user does not have the proper scopes then the tap will throw and exception.
Configure this setting directly using the following Meltano command:
meltano config tap-slack set include_admin_streams [value]Selected Channels (selected_channels)
-
Environment variable:
TAP_SLACK_SELECTED_CHANNELS
A list of channel IDs that should be retrieved. If not defined then all are selected.
Configure this setting directly using the following Meltano command:
meltano config tap-slack set selected_channels [value]Start Date (start_date)
-
Environment variable:
TAP_SLACK_START_DATE
The earliest record date to sync
Configure this setting directly using the following Meltano command:
meltano config tap-slack set start_date [value]Thread Lookback Days (thread_lookback_days)
-
Environment variable:
TAP_SLACK_THREAD_LOOKBACK_DAYS -
Default Value:
1
The number of days to look in the past for new thread replies to existing messages
Configure this setting directly using the following Meltano command:
meltano config tap-slack set thread_lookback_days [value]Expand To Show SDK Settings
Batch Compression Format (batch_config.encoding.compression)
-
Environment variable:
TAP_SLACK_BATCH_CONFIG_ENCODING_COMPRESSION
Compression format to use for batch files.
Configure this setting directly using the following Meltano command:
meltano config tap-slack set batch_config encoding.compression [value]Batch Encoding Format (batch_config.encoding.format)
-
Environment variable:
TAP_SLACK_BATCH_CONFIG_ENCODING_FORMAT
Format to use for batch files.
Configure this setting directly using the following Meltano command:
meltano config tap-slack set batch_config encoding.format [value]Batch Storage Prefix (batch_config.storage.prefix)
-
Environment variable:
TAP_SLACK_BATCH_CONFIG_STORAGE_PREFIX
Prefix to use when writing batch files.
Configure this setting directly using the following Meltano command:
meltano config tap-slack set batch_config storage.prefix [value]Batch Storage Root (batch_config.storage.root)
-
Environment variable:
TAP_SLACK_BATCH_CONFIG_STORAGE_ROOT
Root path to use when writing batch files.
Configure this setting directly using the following Meltano command:
meltano config tap-slack set batch_config storage.root [value]Faker Locale (faker_config.locale)
-
Environment variable:
TAP_SLACK_FAKER_CONFIG_LOCALE
One or more LCID locale strings to produce localized output for: https://faker.readthedocs.io/en/master/#localization
Configure this setting directly using the following Meltano command:
meltano config tap-slack set faker_config locale [value]Faker Seed (faker_config.seed)
-
Environment variable:
TAP_SLACK_FAKER_CONFIG_SEED
Value to seed the Faker generator for deterministic output: https://faker.readthedocs.io/en/master/#seeding-the-generator
Configure this setting directly using the following Meltano command:
meltano config tap-slack set faker_config seed [value]Enable Schema Flattening (flattening_enabled)
-
Environment variable:
TAP_SLACK_FLATTENING_ENABLED
'True' to enable schema flattening and automatically expand nested properties.
Configure this setting directly using the following Meltano command:
meltano config tap-slack set flattening_enabled [value]Max Flattening Depth (flattening_max_depth)
-
Environment variable:
TAP_SLACK_FLATTENING_MAX_DEPTH
The max depth to flatten schemas.
Configure this setting directly using the following Meltano command:
meltano config tap-slack set flattening_max_depth [value]User Stream Map Configuration (stream_map_config)
-
Environment variable:
TAP_SLACK_STREAM_MAP_CONFIG
User-defined config values to be used within map expressions.
Configure this setting directly using the following Meltano command:
meltano config tap-slack set stream_map_config [value]Stream Maps (stream_maps)
-
Environment variable:
TAP_SLACK_STREAM_MAPS
Config object for stream maps capability. For more information check out Stream Maps.
Configure this setting directly using the following Meltano command:
meltano config tap-slack set stream_maps [value]Something missing?
This page is generated from a YAML file that you can contribute changes to.
Edit it on GitHub!Looking for help?
#plugins-general-brightgreen){kind=icon}
-585-c0c0c4){kind=icon}
-14-c0c0c4){kind=icon}
