The tap-rest-api-msdk Singer tap pulls data from REST API that can then be sent to a destination using a Singer target.

Standalone usage #

Install the package using pip: 

pip install tap-rest-api-msdk

For additional instructions, refer to the README in the repository.

Usage with Meltano #

Install Meltano, create your Meltano project, and add the tap to your project as a custom extractor: 

meltano add --custom extractor tap-rest-api-msdk --variant widen

Then, configure the extractor, add any Singer target as a loader to send data to a destination and run a data integration (EL) pipeline.

Capabilities #

Settings #

Name (name) #

(required) name of the stream.

API Base URL (api_url) #

(required) the base url/endpoint for the desired api.

Path (path) #

(optional) the path appeneded to the api_url.

Params (params) #

(optional) an object of objects that provide the params in a requests.get method.

Headers (headers) #

(optional) an object of headers to pass into the api calls.

Records Path (records_path) #

(optional) a jsonpath string representing the path in the requests response that contains the records to process. Defaults to $[*].

Primary Keys (primary_keys) #

(required) a list of the json keys of the primary key for the stream.

Replication Key (replication_key) #

(optional) the json key of the replication key. Note that this should be an incrementing integer or datetime object.

Except Keys (except_keys) #

This tap automatically flattens the entire json structure and builds keys based on the corresponding paths. Keys, whether composite or otherwise, listed in this dictionary will not be recursively flattened, but instead their values will be turned into a json string and processed in that format. This is also automatically done for any lists within the records; therefore, records are not duplicated for each item in lists.

Number Inference Records (num_inference_records) #

(optional) number of records used to infer the stream’s schema. Defaults to 50.

Looking for help? #

If you're having trouble getting tap-rest-api-msdk to work by itself or with Meltano, look for an existing issue in its repository, file a new issue, or join the Meltano Slack community and ask for help in the #plugins-general channel.

Found an issue on this page? #

This page is generated from a YAML file that you can contribute changes to! It is also validated against a JSON Schema used for taps and targets.


Edit this page on GitLab!