Luigi

Luigi: A Simpler, Older Alternative

Luigi 18,778 (github.com/spotify/luigi (https://github.com/spotify/luigi 18,778 )), open-sourced by Spotify in 2012, predates Airflow 129 and is still released (3.8.1, 7 May 2026). A task declares requires() (upstream tasks), output() (a target, usually a file) and run(); it counts as done when its output exists. There is no built-in scheduler: cron starts luigi, and an optional central luigid prevents two workers from running the same task and draws the graph. Luigi 3.8 needs Python 3.13 or older, so demos/ch07/luigi/ uses a python:3.13-slim image:

booknest_luigi.py (excerpt): a task is done when its target existsPython
class Load(luigi.Task):
    ds = luigi.DateParameter()
    def requires(self):
        return Extract(self.ds)
    def output(self):                            # a marker file: "the day is loaded"
        return luigi.LocalTarget(f"{DATA}/luigi/loaded/{self.ds}")
    def run(self):
        orders, lines = p.load_orders(str(self.ds), self.input().path, connect())
        with self.output().open("w") as f:
            f.write(f"{orders} orders, {lines} lines\n")

run.sh 2026-06-30 2026-06-30 ran luigi --module booknest_luigi Publish --ds 2026-06-30 --local-scheduler twice:

Output of 43
Scheduled 3 tasks of which:
* 1 complete ones were encountered:
    - 1 Extract(ds=2026-06-30)
* 2 ran successfully:
    - 1 Load(ds=2026-06-30)
    - 1 Publish(ds=2026-06-30)
Scheduled 1 tasks of which:
* 1 complete ones were encountered:
    - 1 Publish(ds=2026-06-30)

Extract never ran: the Prefect 74,615 runs had landed the file, and Luigi trusts its existence even if the export changed since. That is the model's strength (cheap restarts) and its trap (a target is not a checksum).