Oracle

tap-oracle (s7clarke10 variant)🥈

Cloud Database

The tap-oracle extractor pulls data from Oracle that can then be sent to a destination using a loader.

Alternate Implementations

Getting Started

Prerequisites

If you haven't already, follow the initial steps of the Getting Started guide:

  1. Install Meltano
  2. Create your Meltano project

Installation and configuration

  1. Add the tap-oracle extractor to your project using 
    meltano add
     : 
    2. meltano add extractor tap-oracle
  2. Configure the tap-oracle settings using 
    meltano config
     : 
    3. meltano config tap-oracle set --interactive
  3. Test that extractor settings are valid using 
    meltano config
     : 
    4. meltano config tap-oracle test

Next steps

If you run into any issues, learn how to get help.

Capabilities

The current capabilities for tap-oracle may have been automatically set when originally added to the Hub. Please review the capabilities when using this extractor. If you find they are out of date, please consider updating them by making a pull request to the YAML file that defines the capabilities for this extractor.

This plugin has the following capabilities:

  • catalog
  • discover
  • log-based
  • state

You can override these capabilities or specify additional ones in your meltano.yml by adding the capabilities key.

Settings

The tap-oracle settings that are known to Meltano are documented below. To quickly find the setting you're looking for, click on any setting name from the list:

You can also list these settings using 

meltano config
with the list subcommand: 

meltano config tap-oracle list

You can override these settings or specify additional ones in your meltano.yml by adding the settings key.

Please consider adding any settings you have defined locally to this definition on MeltanoHub by making a pull request to the YAML file that defines the settings for this plugin.

Common Password (common_password)

  • Environment variable: TAP_ORACLE_COMMON_PASSWORD

Support for a common user/password for working with pluggable databases (PDB).


Configure this setting directly using the following Meltano command:

meltano config tap-oracle set common_password [value]

Common Service Name (common_service_name)

  • Environment variable: TAP_ORACLE_COMMON_SERVICE_NAME

Service name when working with pluggable databases (PDB).


Configure this setting directly using the following Meltano command:

meltano config tap-oracle set common_service_name [value]

Common SID (common_sid)

  • Environment variable: TAP_ORACLE_COMMON_SID

SID when working with pluggable databases (PDB).


Configure this setting directly using the following Meltano command:

meltano config tap-oracle set common_sid [value]

Common User (common_user)

  • Environment variable: TAP_ORACLE_COMMON_USER

Support for a common user/password for working with pluggable databases (PDB). Every common user can connect to an perform operations within the root database, and within any PDB in which it has privileges.


Configure this setting directly using the following Meltano command:

meltano config tap-oracle set common_user [value]

Cursor Array Size (cursor_array_size)

  • Environment variable: TAP_ORACLE_CURSOR_ARRAY_SIZE

A numeric setting adjusting the internal buffersize. The common query tuning scenario is for SELECT statements that return a large number of rows over a slow network. Increasing arraysize can improve performance by reducing the number of round-trips to the database. However increasing this value increases the amount of memory required.


Configure this setting directly using the following Meltano command:

meltano config tap-oracle set cursor_array_size [value]

Default Replication Method (default_replication_method)

  • Environment variable: TAP_ORACLE_DEFAULT_REPLICATION_METHOD

The default replication method to use if one is not automatically set on the stream. Options - 'LOG_BASED', 'FULL_TABLE', 'INCREMENTAL'.


Configure this setting directly using the following Meltano command:

meltano config tap-oracle set default_replication_method [value]

Filter Schemas (filter_schemas)

  • Environment variable: TAP_ORACLE_FILTER_SCHEMAS

A comma separated list of schema names to scan. Used to improve the


Configure this setting directly using the following Meltano command:

meltano config tap-oracle set filter_schemas [value]

Filter Tables (filter_tables)

  • Environment variable: TAP_ORACLE_FILTER_TABLES

An array of tables SCHEMA-TABLE1 to select from. Used to imprive the performance of discovery.


Configure this setting directly using the following Meltano command:

meltano config tap-oracle set filter_tables [value]

Host (host)

  • Environment variable: TAP_ORACLE_HOST

Your Oracle host.


Configure this setting directly using the following Meltano command:

meltano config tap-oracle set host [value]

Logminer Call Timeout (logminer_call_timeout)

  • Environment variable: TAP_ORACLE_LOGMINER_CALL_TIMEOUT

The timeout to use when doing log based replication, in milliseconds. Default, None.


Configure this setting directly using the following Meltano command:

meltano config tap-oracle set logminer_call_timeout [value]

Logminer Dynamic Scn Window (logminer_dynamic_scn_window)

  • Environment variable: TAP_ORACLE_LOGMINER_DYNAMIC_SCN_WINDOW

A boolean whether to use a dynamic scan window for log based replication. Default, False.


Configure this setting directly using the following Meltano command:

meltano config tap-oracle set logminer_dynamic_scn_window [value]

Logminer Iter With Reduction Factor (logminer_iter_with_reduction_factor)

  • Environment variable: TAP_ORACLE_LOGMINER_ITER_WITH_REDUCTION_FACTOR

The reduction factor to use when using log based replication. Default, 10.


Configure this setting directly using the following Meltano command:

meltano config tap-oracle set logminer_iter_with_reduction_factor [value]

Offset Value (offset_value)

  • Environment variable: TAP_ORACLE_OFFSET_VALUE

To avoid problems with uncommitted changes being read, you can set offset_value to add to the value found in the STATE for INCREMENTAL loads. If the value provided is for a datetime replication key then the offset_value is read as seconds to offset by, otherwise the value is used as provided. Using offset_value < 0 would result in an overlapping set of records being read each time the tap is run. Using offset_value > 0 may result in data being missed. However it can be useful if a period (month-year) is being used. This prevents the tap from using period >= last-read-period and doubling up on the extract. Usage (offsetting by +1 day in seconds = 24*3600).


Configure this setting directly using the following Meltano command:

meltano config tap-oracle set offset_value [value]

Oracle Python Driver Type / Mode (ora_python_driver_type)

  • Environment variable: TAP_ORACLE_ORA_PYTHON_DRIVER_TYPE
  • Default Value: cx

Determines which Oracle Library Driver and mode is used by tap-oracle. cx=CX_Oracle, thin=oracleDB (No SQL Client), thick=oracleDB (SQL Client). Default, cx.


Configure this setting directly using the following Meltano command:

meltano config tap-oracle set ora_python_driver_type [value]

Password (password)

  • Environment variable: TAP_ORACLE_PASSWORD

Your Oracle password.


Configure this setting directly using the following Meltano command:

meltano config tap-oracle set password [value]

Port (port)

  • Environment variable: TAP_ORACLE_PORT

Your Oracle port.


Configure this setting directly using the following Meltano command:

meltano config tap-oracle set port [value]

Scn Window Size (scn_window_size)

  • Environment variable: TAP_ORACLE_SCN_WINDOW_SIZE

The scan window size when using log based replication. Default, None.


Configure this setting directly using the following Meltano command:

meltano config tap-oracle set scn_window_size [value]

Service Name (service_name)

  • Environment variable: TAP_ORACLE_SERVICE_NAME

Your Oracle service name.


Configure this setting directly using the following Meltano command:

meltano config tap-oracle set service_name [value]

SID (sid)

  • Environment variable: TAP_ORACLE_SID

Your Oracle SID. If connecting directly to an instance, otherwise use service_name.


Configure this setting directly using the following Meltano command:

meltano config tap-oracle set sid [value]

Use Ora Rowscn (use_ora_rowscn)

  • Environment variable: TAP_ORACLE_USE_ORA_ROWSCN

A boolean whether to use ORA_ROWSCN to bookmark progress in case replication is interrupted. ORA_ROWSCN returns an Oracle SCN which is a System Change Number. This is a unique identifier for a particular change (i.e. a transaction). Default, True.


Configure this setting directly using the following Meltano command:

meltano config tap-oracle set use_ora_rowscn [value]

Use Singer Decimal (use_singer_decimal)

  • Environment variable: TAP_ORACLE_USE_SINGER_DECIMAL

This will help avoid numeric rounding issues emitting as a string with a format of singer.decimal.


Configure this setting directly using the following Meltano command:

meltano config tap-oracle set use_singer_decimal [value]

User (user)

  • Environment variable: TAP_ORACLE_USER

Your Oracle user.


Configure this setting directly using the following Meltano command:

meltano config tap-oracle set user [value]

Log Based Replication

For more details about getting log based replication configured, see the Log based replication section of the repo's README.

Something missing?

This page is generated from a YAML file that you can contribute changes to.

Edit it on GitHub!

Looking for help?

If you're having trouble getting the tap-oracle extractor to work, look for an existing issue in its repository, file a new issue, or join the Meltano Slack community and ask for help in the 
#plugins-general
channel.

Install

meltano add extractor tap-oracle

Maintenance Status

  • Maintenance Status

Repo

https://github.com/s7clarke10/pipelinewise-tap-oracle
  • Stars
  • Forks
  • Last Commit Date
  • Open Issues
  • Open PRs
  • Contributors
  • License

Maintainer

  • Steve Clarke

Keywords

  • database