Skip to main content

Firebolt setup

Some core functionality may be limited. If you're interested in contributing, check out the source code for the repository listed below.

  • Maintained by: Firebolt
  • Authors: Firebolt
  • GitHub repo: firebolt-db/dbt-firebolt
  • PyPI package: dbt-firebolt
  • Slack channel: #db-firebolt
  • Supported dbt Core version: v1.1.0 and newer
  • dbt Cloud support: Not Supported
  • Minimum data platform version: n/a

Installing dbt-firebolt

Use pip to install the adapter. Before 1.8, installing the adapter would automatically install dbt-core and any additional dependencies. Beginning in 1.8, installing an adapter does not automatically install dbt-core. This is because adapters and dbt Core versions have been decoupled from each other so we no longer want to overwrite existing dbt-core installations. Use the following command for installation:

Configuring dbt-firebolt

For Firebolt-specific configuration, please refer to Firebolt configs.

For other information including Firebolt feature support, see the GitHub README and the changelog.

Connecting to Firebolt

To connect to Firebolt from dbt, you'll need to add a profile to your profiles.yml file. A Firebolt profile conforms to the following syntax:

profiles.yml
<profile-name>:
target: <target-name>
outputs:
<target-name>:
type: firebolt
user: "<username>"
password: "<password>"
database: "<database-name>"
engine_name: "<engine-name>"
schema: <tablename-prefix>
threads: 1
#optional fields
jar_path: <path-to-local-jdbc-driver>
host: "<hostname>"
account_name: "<account-name>"

Description of Firebolt Profile Fields

To specify values as environment variables, use the format {{ env_var('<variable_name>' }}. For example, {{ env_var('DATABASE_NAME' }}.

FieldDescription
typeThis must be included either in profiles.yml or in the dbt_project.yml file. Must be set to firebolt.
userRequired. A Firebolt username with adequate permissions to access the specified engine_name.
passwordRequired. The password associated with the specified user.
databaseRequired. The name of the Firebolt database to connect to.
engine_nameRequired in version 0.21.10 and later. Optional in earlier versions. The name (not the URL) of the Firebolt engine to use in the specified database. This must be a general purpose read-write engine and the engine must be running. If omitted in earlier versions, the default engine for the specified database is used.
schemaRecommended. A string to add as a prefix to the names of generated tables when using the custom schemas workaround.
threadsRequired. Must be set to 1. Multi-threading is not currently supported.
jar_pathRequired only with versions earlier than 0.21.0. Ignored in 0.21.0 and later. The path to your JDBC driver on your local drive.
hostOptional. The host name of the connection. For all customers it is api.app.firebolt.io, which will be used if omitted.
account_nameRequired if more than one account is associated with the specified user1. Specifies the account name (not the account ID) under which the specified database exists. If omitted, the default account is assumed.

Troubleshooting Connections

If you encounter issues connecting to Firebolt from dbt, make sure the following criteria are met:

  • The engine must be a general-purpose read-write engine, not an analytics engine.
  • You must have adequate permissions to access the engine.
  • The engine must be running.
  • If you're not using the default engine for the database, you must specify an engine name.
  • If there is more than one account associated with your credentials, you must specify an account.

Supporting Concurrent Development

In dbt, database schemas are used to compartmentalize developer environments so that concurrent development does not cause table name collisions. Firebolt, however, does not currently support database schemas (it is on the roadmap). To work around this, we recommend that you add the following macro to your project. This macro will take the schema field of your profiles.yml file and use it as a table name prefix.

-- macros/generate_alias_name.sql
{% macro generate_alias_name(custom_alias_name=none, node=none) -%}
{%- if custom_alias_name is none -%}
{{ node.schema }}__{{ node.name }}
{%- else -%}
{{ node.schema }}__{{ custom_alias_name | trim }}
{%- endif -%}
{%- endmacro %}

For an example of how this works, let’s say Shahar and Eric are both working on the same project.

In her .dbt/profiles.yml, Sharar sets schema=sh, whereas Eric sets schema=er in his. When each runs the customers model, the models will land in the database as tables named sh_customers and er_customers, respectively. When running dbt in production, you would use yet another profiles.yml with a string of your choice.

0