Skip to main content

Deployment environments

Deployment environments in dbt Cloud are crucial for deploying dbt jobs in production and using features or integrations that depend on dbt metadata or results. To execute dbt, environments determine the settings used during job runs, including:

  • The version of dbt Core that will be used to run your project
  • The warehouse connection information (including the target database/schema settings)
  • The version of your code to execute

A dbt Cloud project can have multiple deployment environments, providing you the flexibility and customization to tailor the execution of dbt jobs. You can use deployment environments to create and schedule jobs, enable continuous integration, or more based on your specific needs or requirements.

Learn how to manage dbt Cloud environments

To learn different approaches to managing dbt Cloud environments and recommendations for your organization's unique needs, read dbt Cloud environment best practices.

Learn more about development vs. deployment environments in dbt Cloud Environments.

There are three types of deployment environments that serve different needs:

  • Production: Environment for transforming data and building pipelines for production use.
  • Stagingbeta: Environment for working with production tools while limiting access to production data.
  • General: General use environment for deployment development.

We highly recommend using the Production environment type for the final, source of truth deployment data. There can be only one environment marked for final production workflows and we don't recommend using a General environment for this purpose.

Create a deployment environment​

To create a new dbt Cloud deployment environment, navigate to Deploy -> Environments and then click Create Environment. Select Deployment as the environment type. The option will be greyed out if you already have a development environment.

Navigate to Deploy -> Environments to create a deployment environmentNavigate to Deploy -> Environments to create a deployment environment

Set as production environment​

In dbt Cloud, each project can have one designated deployment environment, which serves as its production environment. This production environment is essential for using features like dbt Explorer and cross-project references. It acts as the source of truth for the project's production state in dbt Cloud.

Set your production environment as the default environment in your Environment SettingsSet your production environment as the default environment in your Environment Settings

Semantic Layer​

For Semantic Layer-eligible customers, the next section of environment settings is the Semantic Layer configurations. The Semantic Layer setup guide has the most up-to-date setup instructions!

Deployment connection​

Warehouse Connections

Warehouse connections are set at the Project level for dbt Cloud accounts, and each Project can have one connection (Snowflake account, Redshift host, Bigquery project, Databricks host, etc.). Some details of that connection (databases/schemas/etc.) can be overridden within this section of the dbt Cloud environment settings.

This section determines the exact location in your warehouse dbt should target when building warehouse objects! This section will look a bit different depending on your warehouse provider.

For all warehouses, use extended attributes to override missing or inactive (grayed-out) settings.

This section will not appear if you are using Postgres, as all values are inferred from the project's connection. Use extended attributes to override these values.

Deployment credentials​

This section allows you to determine the credentials that should be used when connecting to your warehouse. The authentication methods may differ depending on the warehouse and dbt Cloud tier you are on.

Postgres Deployment Credentials SettingsPostgres Deployment Credentials Settings

Editable fields​

  • Username: Postgres username to use (most likely a service account)
  • Password: Postgres password for the listed user
  • Schema: Target schema

Staging environment beta​

note

Currently in limited availability beta. Contact support or your account team if you're interested in beta access.

Staging environments are useful ways to grant developers access to deployment workflows and tools while controlling access to production data. They are configured with their own long-living branch (for example, staging) that may be very similar to main in many ways while potentially limiting the data the developers can access.

Ideally, the workflows would move upstream from the Development environment -> Staging environment -> Production environment with developer branches feeding into the staging branch, then ultimately main. In many cases, the main and staging branches will be identical after a merge and remain until the next batch of changes from the development branches are ready to be elevated. We recommend setting branch protection rules on staging similar to main.

Create a staging environment​

In the dbt Cloud, navigate to Deploy -> Environments and then click Create Environment. Select Deployment as the environment type. The option will be greyed out if you already have a development environment.

Create a staging environmentCreate a staging environment

Follow the steps outlined in deployment credentials to complete the remainder of the environment setup.

We recommend that the data warehouse credentials be for a dedicated user or service principal.

Why use a staging environment​

There are two primary motivations for using a Staging environment:

  1. An additional validation layer before changes are deployed into Production. You can deploy, test, and explore your dbt models in Staging.
  2. Clear isolation between development workflows and production data. It enables developers to work in metadata-powered ways, using features like deferral and cross-project references, without accessing data in production deployments.
Coming soon: environment-level permissions

Provide developers with the ability to create, edit, and trigger ad hoc jobs in the Staging environment, while keeping the Production environment locked down.

Let's say you have Project B downstream of Project A with cross-project refs configured in the models. When developers work in the IDE for Project B, cross-project refs will resolve to the Staging environment of Project A, rather than production. You'll get the same results with those refs when jobs are run in the Staging environment. Only the Production environment will reference the Production data, keeping the data and access isolated without needing separate projects.

If Project B also has a Staging deployment, then references to unbuilt upstream models within Project B will resolve to that environment, using deferral, rather than resolving to the models in Production. This saves developers time and warehouse spend, while preserving clear separation of environments.

Finally, the Staging environment has its own view in dbt Explorer, giving you a full view of your prod and pre-prod data.

Explore in a staging environmentExplore in a staging environment
0
Edit this page