tap-mssql - Meltano Hub

Microsoft SQL Server

tap-mssql (wintersrd variant)🥈

Microsoft SQL Server

The tap-mssql extractor pulls data from Microsoft SQL Server that can then be sent to a destination using a loader.

Alternate Implementations

Getting Started

Prerequisites

If you haven't already, follow the initial steps of the Getting Started guide:

Installation and configuration

Add the tap-mssql extractor to your project using
```
meltano add
```
:

meltano add extractor tap-mssql

Configure the tap-mssql settings using

meltano config

:

meltano config tap-mssql set --interactive

Test that extractor settings are valid using

meltano config

:

meltano config tap-mssql test

Next steps

Follow the remaining steps of the Getting Started guide:

If you run into any issues, learn how to get help.

Capabilities

The current capabilities for tap-mssql may have been automatically set when originally added to the Hub. Please review the capabilities when using this extractor. If you find they are out of date, please consider updating them by making a pull request to the YAML file that defines the capabilities for this extractor.

This plugin has the following capabilities:

catalog
discover
state

You can override these capabilities or specify additional ones in your meltano.yml by adding the capabilities key.

Settings

The tap-mssql settings that are known to Meltano are documented below. To quickly find the setting you're looking for, click on any setting name from the list:

characterset
cursor_array_size
conn_properties
database
enable_tds_logging
filter_dbs
host
password
port
tds_version
use_date_datatype
use_singer_decimal
user

You can also list these settings using

meltano config

with the list subcommand:

meltano config tap-mssql list

You can override these settings or specify additional ones in your meltano.yml by adding the settings key.

Please consider adding any settings you have defined locally to this definition on MeltanoHub by making a pull request to the YAML file that defines the settings for this plugin.

Characterset (characterset)

Environment variable: TAP_MSSQL_CHARACTERSET

The characterset for the database / source system. The default is utf8, however older databases might use a charactersets like cp1252 for the encoding. If you have errors with a UnicodeDecodeError: 'utf-8' codec can't decode byte .... then a solution is to examine the characterset of the source database / system and make an appropriate substitution for utf8 like cp1252.

Configure this setting directly using the following Meltano command:

meltano config tap-mssql set characterset [value]

Cursor Array Size (cursor_array_size)

Environment variable: TAP_MSSQL_CURSOR_ARRAY_SIZE
Default Value: 1

This allows fetching records in batches speeding up network traffic. Recommend a setting of 10000 records.

Configure this setting directly using the following Meltano command:

meltano config tap-mssql set cursor_array_size [value]

Connection Properties (conn_properties)

Environment variable: TAP_MSSQL_CONN_PROPERTIES

The "conn_properties" allows specific configuration of database settings via SQL set statements to send to the database instance upon connection establishment. Can be a string or another kind of iterable of strings. If you want to connect Azure SQL Server instances e.g. PDW you will need to adjust the connection setting for that version of MSSQL. Please refer to the github repo page for details on these settings.

Configure this setting directly using the following Meltano command:

meltano config tap-mssql set conn_properties [value]

Database (database)

Environment variable: TAP_MSSQL_DATABASE

Your MSSQL database name.

Configure this setting directly using the following Meltano command:

meltano config tap-mssql set database [value]

Enable TDS Logging (enable_tds_logging)

Environment variable: TAP_MSSQL_ENABLE_TDS_LOGGING
Default Value: false

When set it will dump out the underlying TDS driver logs. Useful for diagnosing issues if you are having connection issues to SQL Server databases. WARNING! this does dump a lot of information and may log secure data, should be only used in Development environments.

Configure this setting directly using the following Meltano command:

meltano config tap-mssql set enable_tds_logging [value]

Filter Databases (filter_dbs)

Environment variable: TAP_MSSQL_FILTER_DBS

A comma separated list of databases to filter for.

Configure this setting directly using the following Meltano command:

meltano config tap-mssql set filter_dbs [value]

Host (host)

Environment variable: TAP_MSSQL_HOST

Your MSSQL server host.

Configure this setting directly using the following Meltano command:

meltano config tap-mssql set host [value]

Password (password)

Environment variable: TAP_MSSQL_PASSWORD

Your MSSQL password.

Configure this setting directly using the following Meltano command:

meltano config tap-mssql set password [value]

Port (port)

Environment variable: TAP_MSSQL_PORT

The MSSQL port, defaulted to 1433.

Configure this setting directly using the following Meltano command:

meltano config tap-mssql set port [value]

TDS Version (tds_version)

Environment variable: TAP_MSSQL_TDS_VERSION

The TDS version of the connection. Defaults to 7.3, connecting to older databases (ex MS SQL 2005) requires 7.0 or earlier. Please see the pymssql or FreeTSD documentation for more details. Also note, while this is a confusing topic version 7.3 is more current than 8.0.

Configure this setting directly using the following Meltano command:

meltano config tap-mssql set tds_version [value]

Use Date Datatype (use_date_datatype)

Environment variable: TAP_MSSQL_USE_DATE_DATATYPE
Default Value: false

To emit a date datatype as a date without a time component or time without an UTC offset. If this boolean config item is not set, the default behaviour is false i.e. emit date datatypes as a datetime. It is recommended to set this on if you have time datatypes and are having issues uploading into into a target database.

Configure this setting directly using the following Meltano command:

meltano config tap-mssql set use_date_datatype [value]

Use Singer Decimal (use_singer_decimal)

Environment variable: TAP_MSSQL_USE_SINGER_DECIMAL

The ability to output decimal, float, and integer data as strings to avoid loss of precision.

Configure this setting directly using the following Meltano command:

meltano config tap-mssql set use_singer_decimal [value]

User (user)

Environment variable: TAP_MSSQL_USER

Your MSSQL username.

Configure this setting directly using the following Meltano command:

meltano config tap-mssql set user [value]

Something missing?

This page is generated from a YAML file that you can contribute changes to.

Edit it on GitHub!

Looking for help?

If you're having trouble getting the tap-mssql extractor to work, look for an existing issue in its repository, file a new issue, or join the Meltano Slack community and ask for help in the

#plugins-general

channel.

Install

meltano add extractor tap-mssql

Maintenance Status

Repo

https://github.com/wintersrd/pipelinewise-tap-mssql

Maintainer

Meltano Stats

PyPI Stats

Keywords