The tap-mysql Meltano extractor pulls data from MySQL / MariaDB that can then be sent to a destination using a loader.

Alternative variants #

Multiple variants of tap-mysql are available. This document describes the default transferwise variant, which is recommended for new users.

Alternative variants are:

Getting Started #

Prerequisites #

If you haven't already, follow the initial steps of the Getting Started guide:

  1. Install Meltano
  2. Create your Meltano project

Known limitations #

Note that this extractor is incompatible with the default datamill-co variants of target-postgres and target-snowflake, because they don’t support stream names that include the source schema in addition to the table name: <schema>-<table>, e.g. public-accounts.

Instead, use the transferwise variants that were made to be used with this extractor: target-postgres and target-snowflake.

Installation and configuration #

  1. Add the tap-mysql extractor to your project using meltano add : 

    meltano add extractor tap-mysql
  2. Configure the settings below using meltano config .

Next steps #

Follow the remaining steps of the Getting Started guide:

  1. Select entities and attributes to extract
  2. Add a loader to send data to a destination
  3. Run a data integration (EL) pipeline
If you run into any issues, learn how to get help.

Capabilities #

Settings #

tap-mysql requires the configuration of the following settings:

The settings for extractor tap-mysql that are known to Meltano are documented below. To quickly find the setting you're looking for, use the Table of Contents at the top of the page.

Host (host) #

The MySQL hostname.

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config tap-mysql set host <host>

export TAP_MYSQL_HOST=<host>

Port (port) #

The MySQL port.

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config tap-mysql set port 3306

export TAP_MYSQL_PORT=3306

User (user) #

The MySQL username.

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config tap-mysql set user <user>

export TAP_MYSQL_USER=<user>

Password (password) #

The MySQL password.

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config tap-mysql set password <password>

export TAP_MYSQL_PASSWORD=<password>

Database (database) #

The MySQL database name.

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config tap-mysql set database <database>

export TAP_MYSQL_DATABASE=<database>

SSL (ssl) #

Whether the MySQL connection should use SSL.

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config tap-mysql set ssl true

export TAP_MYSQL_SSL=true

Filter DBs (filter_dbs) #

Comma separated list of schemas to extract tables only from particular schemas and to improve data extraction performance

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config tap-mysql set filter_dbs <filter_dbs>

export TAP_MYSQL_FILTER_DBS=<filter_dbs>

Session SQLs (session_sqls) #

  • Environment variable: TAP_MYSQL_SESSION_SQLS
  • Default: SET @@session.time_zone='+0:00'SET @@session.wait_timeout=28800SET @@session.net_read_timeout=3600SET @@session.innodb_lock_wait_timeout=3600

List of SQL commands to run when a connection made. This allows to set session variables dynamically, like timeouts.

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config tap-mysql set session_sqls '[...]'

export TAP_MYSQL_SESSION_SQLS='[...]'

Looking for help? #

If you're having trouble getting the tap-mysql extractor to work, look for an existing issue in its repository, file a new issue, or join the Meltano Slack community and ask for help in the #plugins-general channel.

Found an issue on this page? #

This page is generated from a YAML file that you can contribute changes to. Edit it on GitHub!