BigQuery
Table of Contents
- Alternative variants
- Standalone usage
- Usage with Meltano
- Capabilities
- Settings
-
Project Id (
project_id
) -
Dataset Id (
dataset_id
) -
Location (
location
) -
credentials_path
-
Validate Records (
validate_records
) -
Add Metadata Columns (
add_metadata_columns
) -
Replication Method (
replication_method
) -
Table Prefix (
table_prefix
) -
Table Suffix (
table_suffix
) -
Max Cache (
max_cache
) -
Merge State Messages (
merge_state_messages
) -
Table Config (
table_config
)
-
Project Id (
- Looking for help?
The
target-bigquery
Singer target
sends data into
BigQuery
after it was pulled from a source using a
Singer tap.
Alternative variants
Multiple
variants
of target-bigquery
are available. This document describes the
default adswerve
variant, which is recommended for new
users.
Alternative variants are:
Standalone usage
Install the package using pip:
pip install git+https://github.com/adswerve/target-bigquery.git@v0.11.3
For additional instructions, refer to the README in the repository.
Usage with Meltano
Install Meltano, create your Meltano project, and add the target to your project as a loader:
meltano add loader target-bigquery
For additional instructions, refer to the
Meltano-specific documentation for target-bigquery
.
Capabilities
Settings
Project Id (project_id
)
BigQuery project
Dataset Id (dataset_id
)
BigQuery dataset
Location (location
)
- Default:
US
Dataset location
credentials_path
This is not a real setting, so it should not appear in the config.json
JSON schema. The taret will read the GOOGLE_APPLICATION_CREDENTIALS
environment variable which should be a fully qualified path to client_secrets.json
for your service account.
Validate Records (validate_records
)
- Default:
false
Validate records
Add Metadata Columns (add_metadata_columns
)
- Default:
false
Add _time_extracted
and _time_loaded
metadata columns
Replication Method (replication_method
)
- Default:
append
Replication method, append
or truncate
Table Prefix (table_prefix
)
Add prefix to table name
Table Suffix (table_suffix
)
Add suffix to table name
Max Cache (max_cache
)
- Default:
50
Maximum cache size in MB
Merge State Messages (merge_state_messages
)
- Default:
false
Whether to merge multiple state messages from the tap into the state file or uses the last state message as the state file.
Table Config (table_config
)
A path to a file containing the definition of partitioning and clustering.
Looking for help?
If you're having trouble getting
target-bigquery
to work by itself or with
Meltano, look for an
existing issue in its repository, file a new issue,
or
join the Meltano Slack community
and ask for help in the #plugins-general
channel.
Found an issue on this page?
This page is generated from a YAML file that you can contribute changes to! It is also validated against a JSON Schema used for taps and targets.