dbt source snapshot-freshnesshas been renamed to
dbt source freshness. Its node selection logic is now consistent with other tasks. In order to check freshness for a specific source, you must prefix it with
- Two dbt JSON artifacts have a new schema:
manifest.json(new default node properties) and
sources.json(now includes timing information).
- Snowflake: Turn off transactions and turn on autocommit by default. Explicitly specify
commitfor DML statements in incremental and snapshot materializations. Note that this may affect user-space code that depends on transactions.
New and changed documentation
build, rpc: Add
source: Renamed to
dbt source freshness.
dbt depslogging for outdated packages
--output-keysflag and RPC parameter
source: Updated selection logic to match other tasks. When selecting a specific source to check freshness, you must prefix it with
- Node selection syntax, commands: Switch
--selectacross the board. (Commands which previously used the
--modelsflag still support it for backwards compatibility.)
- YAML selectors now support an optional
defaultproperty. If set, dbt will use custom selection criteria for commands that do not specify their own selection/exclusion flags.
- Selection methods and state comparison caveats: Add
state:modifiedsubselectors, and reflect that it now includes changes to upstream macros.
- Test selection examples includes more discussion of indirect selection (a change in v0.20), and the optional "greedy" flag/property (new in v0.21), which you can optionally set to include tests that have a mix of selected + unselected parents
Elsewhere in Core
- Resource configs and properties docs have been consolidated and reconciled. New
configproperty that makes it possible to configure models, seeds, snapshots, and tests in all yaml files.
- Configuring incremental models: New optional configuration for incremental models,
- Environment variables: Add a log-scrubbing prefix,
whereconfig has been reimplemented as a macro (
get_where_subquery) that you can optionally reimplement, too
dispatchnow supports reimplementing global macros residing in the
dbtmacro namespace with versions from installed packages, by leveraging
- Postgres profile property
connect_timeoutnow configurable. Also applicable to child plugins (e.g.
- Redshift: profile property
ra3: trueto support cross-database source definitions and read-only querying
- BigQuery: profile property
execution_projectnow configurable. Snapshots support