The target-bigquery Singer target sends data into BigQuery after it was pulled from a source using a Singer tap.

Alternative variants

Multiple variants of target-bigquery are available. This document describes the default adswerve variant, which is recommended for new users.

Alternative variants are:

Standalone usage

Install the package using pip: 

pip install git+https://github.com/adswerve/target-bigquery.git@v0.11.3

For additional instructions, refer to the README in the repository.

Usage with Meltano

Install Meltano, create your Meltano project, and add the target to your project as a loader: 

meltano add loader target-bigquery

For additional instructions, refer to the Meltano-specific documentation for target-bigquery.

Capabilities

Settings

Project Id (project_id)

BigQuery project

Dataset Id (dataset_id)

BigQuery dataset

Location (location)

  • Default: US

Dataset location

credentials_path

This is not a real setting, so it should not appear in the config.json JSON schema. The taret will read the GOOGLE_APPLICATION_CREDENTIALS environment variable which should be a fully qualified path to client_secrets.json for your service account.

Validate Records (validate_records)

  • Default: false

Validate records

Add Metadata Columns (add_metadata_columns)

  • Default: false

Add _time_extracted and _time_loaded metadata columns

Replication Method (replication_method)

  • Default: append

Replication method, append or truncate

Table Prefix (table_prefix)

Add prefix to table name

Table Suffix (table_suffix)

Add suffix to table name

Max Cache (max_cache)

  • Default: 50

Maximum cache size in MB

Merge State Messages (merge_state_messages)

  • Default: false

Whether to merge multiple state messages from the tap into the state file or uses the last state message as the state file.

Table Config (table_config)

A path to a file containing the definition of partitioning and clustering.

Looking for help?

If you're having trouble getting target-bigquery to work by itself or with Meltano, look for an existing issue in its repository, file a new issue, or join the Meltano Slack community and ask for help in the #plugins-general channel.

Found an issue on this page?

This page is generated from a YAML file that you can contribute changes to! It is also validated against a JSON Schema used for taps and targets.


Edit this page on GitLab!