MySQL / MariaDB
Table of Contents
The tap-mysql
Meltano extractor pulls data from MySQL / MariaDB that can then be sent to a destination using a loader.
-
- Repository: https://github.com/transferwise/pipelinewise-tap-mysql
-
-
-
-
-
-
-
- Maintainer: Wise
- Meltano Stats (across all plugin variants):
-
-
-
-
Alternative variants #
Multiple
variants
of tap-mysql
are available.
This document describes the default transferwise
variant,
which is recommended for new users.
Alternative variants are:
Getting Started #
Prerequisites #
If you haven't already, follow the initial steps of the Getting Started guide:
Known limitations #
Note that this extractor is incompatible with the default datamill-co
variants
of target-postgres
and target-snowflake
,
because they don’t support stream names that include the source schema in addition to the table name: <schema>-<table>
, e.g. public-accounts
.
Instead, use the transferwise
variants that were made to be used with this extractor:
target-postgres
and target-snowflake
.
Installation and configuration #
-
Add the
tap-mysql
extractor to your project usingmeltano add
:meltano add extractor tap-mysql
-
Configure the settings below using
meltano config
.
Next steps #
Follow the remaining steps of the Getting Started guide:
- Select entities and attributes to extract
- Add a loader to send data to a destination
- Run a data integration (EL) pipeline
Capabilities #
These capabilities can also be overriden
by specifying the capabilities
key in your meltano.yml
file.
Settings #
tap-mysql
requires the
configuration
of the following settings:
The settings for extractor tap-mysql
that are known to Meltano are documented below.
To quickly find the
setting you're looking for, use the Table of Contents at
the top of the page.
You can override these
settings or specify additional
ones in your meltano.yml
by adding the settings
key.
Please consider adding any settings you have defined locally to this definition on MeltanoHub by making a pull
request to the
YAML file
that defines the settings for this
extractor.
Host (host
)
#
-
Environment variable:
TAP_MYSQL_HOST
- Default:
localhost
The MySQL hostname.
How to use #
Manage this setting using
meltano config
or an
environment variable:
meltano config tap-mysql set host <host>
export TAP_MYSQL_HOST=<host>
Port (port
)
#
-
Environment variable:
TAP_MYSQL_PORT
- Default:
3306
The MySQL port.
How to use #
Manage this setting using
meltano config
or an
environment variable:
meltano config tap-mysql set port 3306
export TAP_MYSQL_PORT=3306
User (user
)
#
-
Environment variable:
TAP_MYSQL_USER
The MySQL username.
How to use #
Manage this setting using
meltano config
or an
environment variable:
meltano config tap-mysql set user <user>
export TAP_MYSQL_USER=<user>
Password (password
)
#
-
Environment variable:
TAP_MYSQL_PASSWORD
The MySQL password.
How to use #
Manage this setting using
meltano config
or an
environment variable:
meltano config tap-mysql set password <password>
export TAP_MYSQL_PASSWORD=<password>
Database (database
)
#
-
Environment variable:
TAP_MYSQL_DATABASE
The MySQL database name.
How to use #
Manage this setting using
meltano config
or an
environment variable:
meltano config tap-mysql set database <database>
export TAP_MYSQL_DATABASE=<database>
SSL (ssl
)
#
-
Environment variable:
TAP_MYSQL_SSL
- Default:
false
Whether the MySQL connection should use SSL.
How to use #
Manage this setting using
meltano config
or an
environment variable:
meltano config tap-mysql set ssl true
export TAP_MYSQL_SSL=true
Filter DBs (filter_dbs
)
#
-
Environment variable:
TAP_MYSQL_FILTER_DBS
Comma separated list of schemas to extract tables only from particular schemas and to improve data extraction performance
How to use #
Manage this setting using
meltano config
or an
environment variable:
meltano config tap-mysql set filter_dbs <filter_dbs>
export TAP_MYSQL_FILTER_DBS=<filter_dbs>
Session SQLs (session_sqls
)
#
-
Environment variable:
TAP_MYSQL_SESSION_SQLS
- Default:
SET @@session.time_zone='+0:00'SET @@session.wait_timeout=28800SET @@session.net_read_timeout=3600SET @@session.innodb_lock_wait_timeout=3600
List of SQL commands to run when a connection made. This allows to set session variables dynamically, like timeouts.
How to use #
Manage this setting using
meltano config
or an
environment variable:
meltano config tap-mysql set session_sqls '[...]'
export TAP_MYSQL_SESSION_SQLS='[...]'
Looking for help? #
If you're having trouble getting the
tap-mysql
extractor to work, look for an
existing issue in its repository, file a new issue,
or
join the Meltano Slack community
and ask for help in the #plugins-general
channel.
Found an issue on this page? #
This page is generated from a YAML file that you can contribute changes to. Edit it on GitHub!