Software-Defined Assets

A software-defined asset is a function decorated with @asset whose return value, or side effect, is a named data object: a file, a table, a model. Its key (landing/orders_file), its upstream assets, its partitions and its code live together, so Dagster 177,056 can draw lineage, tell you which assets are stale and materialize any subset in dependency order. BookNest's ingest step becomes two definitions on daily partitions:

definitions.py (excerpt): daily partitions and the ingest assetsPython

daily = dg.DailyPartitionsDefinition(start_date="2026-06-28", end_date="2026-07-01",
                                     timezone="UTC")           # 28-30 June (end exclusive)
@dg.asset(partitions_def=daily, key_prefix="landing", group_name="ingest")
def orders_file(context: dg.AssetExecutionContext) -> dg.MaterializeResult:
    ds = context.partition_key
    path, n = p.extract_orders(ds, f"{DATA}/source/orders.jsonl", f"{DATA}/landing")
    return dg.MaterializeResult(metadata={"path": path, "orders": n})
@dg.multi_asset(partitions_def=daily, group_name="ingest", specs=[
    dg.AssetSpec(["shop", "orders"], deps=[orders_file]),     # = dbt's source('shop', ...)
    dg.AssetSpec(["shop", "order_items"], deps=[orders_file])])
def shop_tables(context: dg.AssetExecutionContext, db: BookNestDB):
    ds = context.partition_key
    landed = f"{DATA}/landing/orders/date={ds}/orders.jsonl"
    orders, lines = p.load_orders(ds, landed, db.connect())
    yield dg.MaterializeResult(asset_key=["shop", "orders"], metadata={"rows": orders})
    yield dg.MaterializeResult(asset_key=["shop", "order_items"], metadata={"rows": lines})

A partition is Dagster's ds: each of the three days is materialized, tracked and re-run on its own. multi_asset declares two tables written by one function, because load_orders replaces both in one transaction. Row-count metadata is charted per run. dagster asset list printed 14 keys, from landing/orders_file and shop/orders to the dbt 37,942 models of dagster-dbt and mart/daily_genre_sales.

An asset that returns data hands it to an I/O manager to store; BookNest's assets return only metadata and leave the data in PostgreSQL 1,289 , the usual choice for SQL pipelines.

MENU