Triggerer and Task Execution

The Triggerer, Task Isolation and the Task Execution Interface

A task that waits for a file, a remote job or a time wastes a worker slot while it sleeps. A deferrable operator hands the wait to the triggerer instead: the task releases its slot and registers a trigger, a small async coroutine. The triggerer runs many of them in one asyncio event loop (default capacity 1,000) and, when one fires, the scheduler queues the task again to resume. Sensors and Task Mapping writes one.

Airflow 3 129 's bigger change is task isolation. In Airflow 2 a task imported Airflow's internals and opened its own connection to the metadata database. In Airflow 3, task code runs through the Task SDK (airflow.sdk, version 1.3.2 alongside Airflow 3.3.2) and never reaches the database. A supervisor process starts the task as a child and relays every runtime interaction (claiming the task instance, heartbeats, connections, variables, XComs, the final state) through the Task Execution API on the API server, authenticated by a short-lived JWT for that one task instance. The supervisor keeps heartbeating even while your code blocks.

This interface lets workers run where the database is unreachable (the EdgeExecutor), lets the Task SDK version independently of the server, and opens the door to tasks in other languages (The Task SDK). It also means task code can no longer open airflow.settings.Session and query tables; such DAGs must move to the REST API.