dbt CLI: Create a project

Now that we've successfully run our sample query in BigQuery, and chosen the way we want to develop, we can create a dbt project! In this step, we'll create a starter project with example models, before we build our own models.

info

These are the instructions for developing a project using the dbt CLI. If you're developing in dbt Cloud, follow the instructions here.

Create a repository

We're going to use GitHub as our git provider for this tutorial, but you can use any git provider. If you don't yet have a GitHub account, create one now.

  1. Create a new GitHub repository here named dbt-tutorial.
  2. Click Create repository (without .gitignore and without a license).
Create a GitHub repo

Create a GitHub repo

Create a project

The dbt CLI comes with a command to help you scaffold a dbt project. To create your dbt project:

  1. Ensure dbt is installed by running dbt --version:
$ dbt --version
info

dbt should have been installed as part of the Setting Up part of the tutorial. If it was not installed, please follow the installation instructions

  1. Run the init command:
$ dbt init jaffle-shop
  1. cd into your project:
$ cd jaffle-shop

You can use pwd to confirm that you are in the right spot.

  1. Open your project (i.e. the directory you just created) in a code editor like Atom or VSCode. You should see a directory structure with .sql and .yml files that were generated by the init command.
The starter project in a code editor

The starter project in a code editor

  1. Update the following values in the dbt_project.yml file:
dbt_project.yml
name: jaffle_shop # this normally says my_new_package
...
profile: jaffle_shop # this normally says default
...
models:
jaffle_shop: #this normally says my_new_package. It should match the value for `name:`
...

Connect to BigQuery

When developing locally, dbt connects to your data warehouse using a profile — a yaml file with all the connection details to your warehouse.

  1. Create a file in the ~/.dbt/ directory named profiles.yml.
  2. Move your BigQuery keyfile into this directory.
  3. Copy the following into the file — make sure you update the values where indicated.
profiles.yml
jaffle_shop: # this needs to match the profile: in your dbt_project.yml file
target: dev
outputs:
dev:
type: bigquery
method: service-account
keyfile: /Users/claire/.dbt/dbt-user-creds.json # replace this with the full path to your keyfile
project: grand-highway-265418 # Replace this with your project id
dataset: dbt_alice # Replace this with dbt_your_name, e.g. dbt_bob
threads: 1
timeout_seconds: 300
location: US
priority: interactive
  1. Execute the debug command from your project to confirm that you can successfully connect
$ dbt debug

Confirm that the last line of the output is Connection test: OK connection ok.

A successful dbt debug command

A successful dbt debug command

FAQs

 My data team uses a different data warehouse. What should my profiles.yml file look like for my warehouse?
 Why are profiles stored outside of my project?
 What should I name my profile?
 What should I name my target?
 Can I use environment variables in my profile?

Perform your first dbt run

Our sample project has some example models in it. We're going to check that we can run them to confirm everything is in order.

  1. Execute the run command to build the example models:
$ dbt run

You should have an ouput that looks like this:

A successful dbt run command

A successful dbt run command

Commit your changes

We need to commit our changes so that our repository has up-to-date code.

  1. Link the GitHub repository you created to your dbt project by running the following commands. Make sure you use the correct git URL for your repository.
$ git init
$ git commit -m "Create a dbt project"
$ git remote add origin https://github.com/USERNAME/dbt-tutorial.git
$ git push -u origin master
info

If this is your first time using git, it's worth taking some time to understand the basics.