SQLFluff

sqlfluff from sqlfluff

The sqlfluff utility is a linting tool for SQL files, often used with dbt to enforce SQL code standards.

Getting Started

Prerequisites

If you haven't already, follow the initial steps of the Getting Started guide:

  1. Install Meltano
  2. Create your Meltano project

Installation and configuration

  1. Add the sqlfluff utility to your project using
    meltano add
    :
  2. meltano add utility sqlfluff
  3. Configure the sqlfluff settings using
    meltano config
    :
  4. meltano config sqlfluff set --interactive

Next steps

  1. Update the pip_url in your meltano.yml by appending your dbt adapter (e.g. dbt-snowflake, etc.).

    pip_url: sqlfluff sqlfluff-templater-dbt dbt-core dbt-snowflake
    
  2. Re-install the plugin with the updated pip_url:

    meltano install utility sqlfluff
    
  3. Create a .sqlfluff file in the root directory of your project using the sample content below. You will need to put your dbt adapter name in for dialect and profiles_dir without the dbt prefix (e.g. snowflake). This config file is where you can customize the linting rules to match your team's style guide. SQLFluff comes with logic defaults so you will only need to override configurations you want to change.

    [sqlfluff]
    templater = dbt
    dialect = <your_dialect_name>
    
    [sqlfluff:templater:dbt]
    project_dir = transform
    profiles_dir = transform/profiles/<your_dialect_name>
    profile = meltano
    
  4. Create a .sqlfluffignore in the root directory of your project using the sample content below. This makes sure SQLFluff ignores auto generated sql or installed packages.

    .meltano/
    utilities/
    transform/dbt_packages/
    transform/target/
    transform/dbt_modules/
    transform/macros/
    
  5. Depending on your dbt adapter you will need to override your dbt environment variables using the env key so when SQLFluff calls dbt your profile.yml env vars are properly set. Refer to the Meltano transformer docs for details on what variables are needed for your adapter. An example for Snowflake is shown below.

    utilities:
    - name: sqlfluff
      variant: sqlfluff
      pip_url: sqlfluff sqlfluff-templater-dbt dbt-core dbt-snowflake
      settings:
      - name: user
        env: DBT_SNOWFLAKE_USER
      - name: password
        kind: password
        env: DBT_SNOWFLAKE_PASSWORD
      - name: role
        env: DBT_SNOWFLAKE_ROLE
      - name: account
        env: DBT_SNOWFLAKE_ACCOUNT
      - name: warehouse
        env: DBT_SNOWFLAKE_WAREHOUSE
      - name: schema
        env: DBT_SNOWFLAKE_SCHEMA
        value: foo # This isnt used so we just put a placeholder by default
      - name: database
        env: DBT_SNOWFLAKE_DATABASE
        value: foo # This isnt used so we just put a placeholder by default
    
  6. SqlFluff does still need access to your warehouse so you have to supply valid credentials similar to your dbt configs which will live under your meltano.yml config and .env file.

    meltano config sqlfluff set user <your username>
    meltano config sqlfluff set account <your account>
    meltano config sqlfluff set role <your role>
    meltano config sqlfluff set warehouse <your warehouse>
    meltano config sqlfluff set password <your password>
    
  7. Run meltano invoke sqlfluff:lint to lint your SQL files.

  8. Run meltano invoke sqlfluff:fix to automatically fix your SQL files based on your linting rules.

If you run into any issues, learn how to get help.

Capabilities

This plugin currently has no capabilities defined. If you know the capabilities required by this plugin, please contribute!

Settings

This plugin currently has no settings defined. If you know the settings required by this plugin, please contribute!

Commands

The sqlfluff utility supports the following commands that can be used with
meltano invoke
:

fix

  • Equivalent to: fix

Automatically fix SQL files according to your defined rules.

meltano invoke sqlfluff:fix [args...]

fix-force

  • Equivalent to: fix --force

Automatically fix SQL files according to your defined rules (does not ask for confirmation).

meltano invoke sqlfluff:fix-force [args...]

lint

  • Equivalent to: lint

Lint SQL in transform models.

meltano invoke sqlfluff:lint [args...]

Something missing?

This page is generated from a YAML file that you can contribute changes to.

Edit it on GitHub!

Looking for help?

If you're having trouble getting the sqlfluff utility to work, look for an existing issue in its repository, file a new issue, or join the Meltano Slack community and ask for help in the
#plugins-general
channel.

Install

meltano add utility sqlfluff

Maintenance Status

  • Maintenance Status
  • Stars
  • Forks
  • Open Issues
  • Open PRs
  • Contributors
  • License

Maintainer

  • SQLFluff

Meltano Stats

  • Total Executions (Last 3 Months)
  • Projects (Last 3 Months)

Keywords