Measured on this shared 4-CPU host, each with BookNest's pipeline deployed and idle after its runs:
| Airflow 3.3.2 129 (LocalExecutor) | Dagster 177,056 (dagster dev) | Prefect 74,615 (server, worker, DB) | |
|---|---|---|---|
| Containers | 5 | 1 (5 processes) | 3 |
| Memory | 1,474 MiB | 615-680 MiB | 410 MiB |
| One day's run | 26 s | 100-160 s (14 s in process) | 11.5 s (17 s from pick-up) |
| Metadata store | PostgreSQL 1,289 or MySQL 524 | SQLite 4,756 , PostgreSQL, MySQL | SQLite or PostgreSQL |
dbt 37,942 took 10-12 seconds in every run; the rest is orchestration. Airflow pays for a scheduler, a DAG processor, a triggerer and an API server that all poll, plus a process per task; Dagster's default executor pays a Python start-up per step; Prefect's runs are one process. In production Airflow needs the same five services, while Dagster and Prefect add daemons, Redis 2,763 or more servers as they grow.
To learn, Prefect needs decorators and deployments; Airflow adds scheduling semantics (data intervals, catch-up, XComs, providers); Dagster asks for the biggest shift, to assets, partitions and resources, and repays it in lineage and testability. All three are Apache 2.0, with paid hosted versions (Astronomer 149,584 and the cloud providers' managed Airflow, Dagster+, Prefect Cloud) covered in Managed Orchestration.