The tap-mysql extractor pulls data from MySQL/MariaDB that can then be sent to a destination using a loader.
Alternate Implementations
- Airbyte🥈
- Stitch Data🥈
- Wise (default)🥈
Getting Started
Prerequisites
If you haven't already, follow the initial steps of the Getting Started guide:
Known limitations
Note that this extractor is incompatible with the datamill-co
variants
of target-postgres
and target-snowflake
,
because they don't support stream names that include the source schema in addition to the table name: <schema>-<table>
, e.g. public-accounts
.
Installation and configuration
-
Add the tap-mysql extractor to your
project using
:meltano add
-
Configure the tap-mysql
settings using
:meltano config
-
Test that extractor settings are valid using
:meltano config
meltano add extractor tap-mysql
meltano config tap-mysql set --interactive
meltano config tap-mysql test
Next steps
Follow the remaining steps of the Getting Started guide:
If you run into any issues, learn how to get help.
Capabilities
The current capabilities for
tap-mysql
may have been automatically set when originally added to the Hub. Please review the
capabilities when using this extractor. If you find they are out of date, please
consider updating them by making a pull request to the YAML file that defines the
capabilities for this extractor.
This plugin has the following capabilities:
- catalog
- discover
- state
You can
override these capabilities or specify additional ones
in your meltano.yml
by adding the capabilities
key.
Settings
The
tap-mysql
settings that are known to Meltano are documented below. To quickly
find the setting you're looking for, click on any setting name from the list:
host
port
user
password
database
cursorclass
server_id
filter_dbs
use_gtid
engine
ssl
ssl_ca
ssl_cert
ssl_key
internal_hostname
session_sqls
You can also list these settings using
with the meltano config
list
subcommand:
meltano config tap-mysql list
You can
override these settings or specify additional ones
in your meltano.yml
by adding the settings
key.
Please consider adding any settings you have defined locally to this definition on MeltanoHub by making a pull request to the YAML file that defines the settings for this plugin.
Host (host)
-
Environment variable:
TAP_MYSQL_HOST
-
Default Value:
localhost
The MySQL/MariaDB hostname.
Configure this setting directly using the following Meltano command:
meltano config tap-mysql set host [value]
Port (port)
-
Environment variable:
TAP_MYSQL_PORT
-
Default Value:
3306
The MySQL/MariaDB port.
Configure this setting directly using the following Meltano command:
meltano config tap-mysql set port [value]
User (user)
-
Environment variable:
TAP_MYSQL_USER
The MySQL/MariaDB username.
Configure this setting directly using the following Meltano command:
meltano config tap-mysql set user [value]
Password (password)
-
Environment variable:
TAP_MYSQL_PASSWORD
The MySQL/MariaDB password.
Configure this setting directly using the following Meltano command:
meltano config tap-mysql set password [value]
Database (database)
-
Environment variable:
TAP_MYSQL_DATABASE
The MySQL/MariaDB database name.
Configure this setting directly using the following Meltano command:
meltano config tap-mysql set database [value]
Cursor Class (cursorclass)
-
Environment variable:
TAP_MYSQL_CURSORCLASS
Cursor class used by PyMYSQL.
Configure this setting directly using the following Meltano command:
meltano config tap-mysql set cursorclass [value]
Server ID (server_id)
-
Environment variable:
TAP_MYSQL_SERVER_ID
Used as the slave id when this tap is connecting to the server.
Configure this setting directly using the following Meltano command:
meltano config tap-mysql set server_id [value]
Filter DBs (filter_dbs)
-
Environment variable:
TAP_MYSQL_FILTER_DBS
Comma separated list of schemas to extract tables only from particular schemas and to improve data extraction performance.
Configure this setting directly using the following Meltano command:
meltano config tap-mysql set filter_dbs [value]
Use GTID (use_gtid)
-
Environment variable:
TAP_MYSQL_USE_GTID
-
Default Value:
false
Flag to enable log based replication using GTID.
Configure this setting directly using the following Meltano command:
meltano config tap-mysql set use_gtid [value]
Engine (engine)
-
Environment variable:
TAP_MYSQL_ENGINE
-
Default Value:
mysql
Indicate which flavor the server is, used for LOG_BASED with GTID. Can be either 'mysql' or 'mariadb'.
Configure this setting directly using the following Meltano command:
meltano config tap-mysql set engine [value]
SSL (ssl)
-
Environment variable:
TAP_MYSQL_SSL
-
Default Value:
false
Whether the MySQL/MariaDB connection should use SSL.
Configure this setting directly using the following Meltano command:
meltano config tap-mysql set ssl [value]
SSL CA (ssl_ca)
-
Environment variable:
TAP_MYSQL_SSL_CA
CA for self-signed SSL.
Configure this setting directly using the following Meltano command:
meltano config tap-mysql set ssl_ca [value]
SSL Certificate (ssl_cert)
-
Environment variable:
TAP_MYSQL_SSL_CERT
Certificate for self-signed SSL.
Configure this setting directly using the following Meltano command:
meltano config tap-mysql set ssl_cert [value]
SSL Key (ssl_key)
-
Environment variable:
TAP_MYSQL_SSL_KEY
Key for self-signed SSL.
Configure this setting directly using the following Meltano command:
meltano config tap-mysql set ssl_key [value]
Internal Hostname (internal_hostname)
-
Environment variable:
TAP_MYSQL_INTERNAL_HOSTNAME
Override match hostname for Google cloud.
Configure this setting directly using the following Meltano command:
meltano config tap-mysql set internal_hostname [value]
Session SQLs (session_sqls)
-
Environment variable:
TAP_MYSQL_SESSION_SQLS
-
Default Value:
["SET @@session.time_zone='+0:00'","SET @@session.wait_timeout=28800","SET @@session.net_read_timeout=3600","SET @@session.innodb_lock_wait_timeout=3600"]
List of SQL commands to run when a connection made. This allows to set session variables dynamically, like timeouts.
Configure this setting directly using the following Meltano command:
meltano config tap-mysql set session_sqls [value]
Something missing?
This page is generated from a YAML file that you can contribute changes to.
Edit it on GitHub!Looking for help?
#plugins-general
channel.