dbt Tests Revisited

dbt Tests Revisited

dbt Tests on Lakehouse Models, Revisited

Generic and Singular Tests's generic and singular tests run on the lakehouse once dbt 37,942 reaches it: quality/dbt_lake uses dbt-duckdb, whose profile loads httpfs and iceberg, creates the S3 secret and attaches the REST catalog (type: iceberg). The model fct_order_lifecycle turns the events into one row per order (placed_at, paid_at, shipped_at, delivered_at, last_event):

_models.yml: tests on the lifecycle model built from the Iceberg event tableYAML

version: 2
models:
  - name: fct_order_lifecycle
    columns:
      - name: order_id
        data_tests:
          - unique
          - not_null
          - relationships:
              name: lifecycle_order_exists_in_orders
              arguments: {to: "source('lake', 'orders')", field: order_id}
      - name: last_event
        data_tests:
          - accepted_values:
              name: lifecycle_last_event_valid
              arguments:
                values: [order_placed, order_paid, order_shipped, order_delivered,
                         order_cancelled, order_returned]

Output

2 of 6 PASS assert_daily_sales_reconciles
1 of 6 OK created sql view model main.fct_order_lifecycle
3 of 6 PASS lifecycle_last_event_valid
5 of 6 PASS not_null_fct_order_lifecycle_order_id
6 of 6 PASS unique_fct_order_lifecycle_order_id
4 of 6 FAIL 900 lifecycle_order_exists_in_orders
Got 900 results, configured to fail if != 0
Done. PASS=5 WARN=0 ERROR=1 SKIP=0 NO-OP=0 REUSED=0 TOTAL=6

The singular test assert_daily_sales_reconciles confirms that daily_sales sums to the 100,000 orders; the relationships test finds Soda's 900 stream-only orders at order grain. The model is a DuckDB 61,228 view over Iceberg 129 : materializing a table in the lake catalog failed (This table (probe_table__dbt_tmp) was modified already, can't be renamed!); dbt-trino and dbt-spark write Iceberg tables directly. config: {severity: warn} turns a known gap into a warning, and store_failures: true keeps the failing rows.

MENU