Syncing dbt Models
You can import existing dbt models as Fact Sources by adding tags to your dbt configuration file. This helps Eppo stay in sync with changes in your dbt project.
This page discusses integrating with dbt models, not dbt's semantic layer. If you are interested in integrating with a semantic layer, please reach out to support@geteppo.com
Adding tags to dbt models
To specify a dbt model as an Eppo Fact Source, first add a tag eppo_fact_source
to the model specification to indicate that the model should be interpreted as an Eppo Fact Source:
version: 2
models:
- name: revenue
description: "Central Revenue Model"
tags:
- eppo_fact_source
Next, specify what columns correspond to timestamps, entities, facts, and properties using the corresponding tags:
version: 2
models:
- name: revenue
description: "Central Revenue Model"
tags:
- eppo_fact_source
columns:
- name: anonymous_id
tags:
- eppo_entity:Anonymous User
- name: purchase_timestamp
tags:
- eppo_timestamp
- name: gross_revenue
description: "The gross revenue amount"
tags:
- eppo_fact
- name: event_name
description: "The name of the event"
tags:
- eppo_property
Note that to specify that a column corresponds with an Eppo entity, use the format:
tags:
- eppo_entity:{name of entity in the UI}
Syncing
Eppo provides a python package to sync metrics to your Eppo workspace. This section walks through syncing your dbt models either manually from your local environment, or as part of a CI/CD workflow that automatically keeps metrics in sync with your existing version control system (e.g., GitHub).
From your local environment
To start, you'll need an Eppo API key with read/write Certified Metrics access. This can be created in the Eppo UI by navigating to Admin >> API Keys.
Next, install the eppo_metrics_sync package and set local environment variables:
python3 -m pip install eppo_metrics_sync
export EPPO_API_KEY=<your API key>
export EPPO_SYNC_TAG=local_dev_sync
Now, call the eppo_metrics_sync module and point it at a directory of yaml files (replacing my_database.my_schema
with the location of your data models within the warehouse):
python3 -m eppo_metrics_sync models --schema=dbt-model --dbt-model-prefix="my_database.my_schema"
Note that this assumes the dbt yaml files live in the models
directory. If they live somewhere in your repository, simply replace models
with the path to your dbt yaml files.
You should now see your dbt models in your Eppo workspace under Definitions >> Facts tab. To push updated fact source definitions, simply call the eppo_metrics_sync module again.
If you would like an additional workspace for testing, please reach out to support@geteppo.com
From a GitHub repository
We recommend setting up a separate Eppo workspace for staging changes to Certified Metrics. Please contact your Eppo support representative or email support@geteppo.com to have this created.
To connect a GitHub repository, you’ll need to complete the following steps:
- Add tags to your dbt model configuration files as described above
- Create an API key in both your production and staging Eppo workspaces. This can be done by going to Admin >> API Keys in each workspace. Make sure that the keys have read/write access to the Certified Metrics Sync permission
- Add the API keys as a GitHub secrets named
EPPO_API_KEY
andEPPO_API_KEY_STAGING
to your dbt project. (Alternatively, you can use GitHub environments, but will need to adjust the workflow yaml below slightly) - Copy the following GitHub workflow yaml into a new file
.github/workflows/run_eppo_metric_sync.yaml
. Replacemy_database.my_schema
with the location of your dbt models within your data warehouse.
name: Sync Eppo Metrics
on:
pull_request:
branches:
- main
push:
branches:
- main
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Set up Python 3.10
uses: actions/setup-python@v3
with:
python-version: "3.10"
- name: Install dependencies
run: |
python3 -m pip install --upgrade pip
python3 -m pip install eppo_metrics_sync
- name: Sync Eppo Metrics (Prod)
env:
EPPO_API_KEY: ${{ secrets.EPPO_API_KEY }}
EPPO_SYNC_TAG: demo_company_metric_repository
if: github.event_name != 'pull_request'
run: |
python3 -m eppo_metrics_sync models --schema=dbt-model --dbt-model-prefix="my_database.my_schema"
- name: Sync Eppo Metrics (Staging)
env:
EPPO_API_KEY: ${{ secrets.EPPO_API_KEY_STAGING }}
EPPO_SYNC_TAG: demo_company_metric_repository
run: |
python3 -m eppo_metrics_sync models --schema=dbt-model --dbt-model-prefix="my_database.my_schema"
- Push this to your new GitHub branch and confirm that the new metrics are showing up in your staging environment. You can view the metric sync logs on the Admin page under Metric Sync Log:
- Once you are happy with your new fact sources, merge the branch to main and you should see your new certified fact sources in your production Eppo environment!