The tap-mysql Meltano extractor pulls data from MySQL / MariaDB that can then be sent to a destination using a loader.

Alternative variants #

Multiple variants of tap-mysql are available. This document describes the default transferwise variant, which is recommended for new users.

Alternative variants are:

Getting Started #

Prerequisites #

If you haven't already, follow the initial steps of the Getting Started guide:

  1. Install Meltano
  2. Create your Meltano project

Known limitations #

Note that this extractor is incompatible with the default datamill-co variants of target-postgres and target-snowflake, because they don’t support stream names that include the source schema in addition to the table name: <schema>-<table>, e.g. public-accounts.

Instead, use the transferwise variants that were made to be used with this extractor: target-postgres and target-snowflake.

Installation and configuration #

  1. Add the tap-mysql extractor to your project using meltano add : 

    meltano add extractor tap-mysql
  2. Configure the settings below using meltano config .

Next steps #

Follow the remaining steps of the Getting Started guide:

  1. Select entities and attributes to extract
  2. Add a loader to send data to a destination
  3. Run a data integration (EL) pipeline
If you run into any issues, learn how to get help.

Capabilities #

These capabilities can also be overriden by specifying the capabilities key in your meltano.yml file.

Settings #

tap-mysql requires the configuration of the following settings:

The settings for extractor tap-mysql that are known to Meltano are documented below. To quickly find the setting you're looking for, use the Table of Contents at the top of the page.

You can override these settings or specify additional ones in your meltano.yml by adding the settings key. Please consider adding any settings you have defined locally to this definition on MeltanoHub by making a pull request to the YAML file that defines the settings for this extractor.

Host (host) #

The MySQL hostname.

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config tap-mysql set host <host>

export TAP_MYSQL_HOST=<host>

Port (port) #

The MySQL port.

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config tap-mysql set port 3306

export TAP_MYSQL_PORT=3306

User (user) #

The MySQL username.

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config tap-mysql set user <user>

export TAP_MYSQL_USER=<user>

Password (password) #

The MySQL password.

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config tap-mysql set password <password>

export TAP_MYSQL_PASSWORD=<password>

Database (database) #

The MySQL database name.

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config tap-mysql set database <database>

export TAP_MYSQL_DATABASE=<database>

SSL (ssl) #

Whether the MySQL connection should use SSL.

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config tap-mysql set ssl true

export TAP_MYSQL_SSL=true

Filter DBs (filter_dbs) #

Comma separated list of schemas to extract tables only from particular schemas and to improve data extraction performance

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config tap-mysql set filter_dbs <filter_dbs>

export TAP_MYSQL_FILTER_DBS=<filter_dbs>

Session SQLs (session_sqls) #

  • Environment variable: TAP_MYSQL_SESSION_SQLS
  • Default: SET @@session.time_zone='+0:00'SET @@session.wait_timeout=28800SET @@session.net_read_timeout=3600SET @@session.innodb_lock_wait_timeout=3600

List of SQL commands to run when a connection made. This allows to set session variables dynamically, like timeouts.

How to use #

Manage this setting using meltano config or an environment variable: 

meltano config tap-mysql set session_sqls '[...]'

export TAP_MYSQL_SESSION_SQLS='[...]'

Looking for help? #

If you're having trouble getting the tap-mysql extractor to work, look for an existing issue in its repository, file a new issue, or join the Meltano Slack community and ask for help in the #plugins-general channel.

Found an issue on this page? #

This page is generated from a YAML file that you can contribute changes to. Edit it on GitHub!