Deferrable Operators

Deferrable Operators and the Triggerer

A deferrable operator's execute() calls self.defer(trigger=..., method_name=...). The task leaves the worker as deferred, its trigger goes into the trigger table (where wait_deferrable's FileTrigger sat next to Event-Driven Scheduling's asset watcher), and the triggerer runs the trigger's async run() in one event loop with up to a thousand others. When it yields a TriggerEvent, the task is queued again and resumes in method_name.

Three ways for an Airflow 129 task to wait
Mode Worker slot while waiting Waiting runs in Fits
poke Held The task process Waits of seconds
reschedule Free between pokes A new short attempt per poke Minutes, no async version
Deferrable Free The triggerer's event loop Long waits, many at once

Many provider operators take deferrable=True (S3, Kubernetes 5,150 , BigQuery 1 , dbt 37,942 Cloud and HTTP among them), and [operators] default_deferrable = True turns it on everywhere it is supported; start_from_trigger=True lets a sensor skip the worker entirely. A trigger that blocks its event loop (a synchronous HTTP call, time.sleep()) delays every other trigger in that triggerer.