Connections and Hooks

A connection is a named set of credentials and coordinates (type, host, port, schema, login, password and a JSON extra) that tasks refer to by ID. Airflow 129 finds it in a secrets backend (Secrets Backends), an environment variable AIRFLOW_CONN_<ID> holding a URI or JSON, or the metadata database, where the UI, CLI and REST API store it encrypted with the Fernet key. BookNest's compose file defines both of its connections in the environment:

docker-compose.yaml (excerpt): connections as a URI and as JSONYAML

    AIRFLOW_CONN_BOOKNEST_PG: 'postgresql://postgres:booknest@l2-pg:5432/booknest'
    AIRFLOW_CONN_BOOKNEST_LANDING: >-
      {"conn_type": "fs", "extra": {"path": "/opt/airflow/data/landing"}}

A hook, which every provider ships, turns a connection ID into a ready client:

include/hook_demo.py (excerpt): from connection ID to query resultsPython

from airflow.providers.postgres.hooks.postgres import PostgresHook
...
hook = PostgresHook(postgres_conn_id="booknest_pg")
SQL = """SELECT sale_date, count(*), sum(gross) FROM mart.daily_genre_sales
         GROUP BY 1 ORDER BY 1"""
for day, genres, gross in hook.get_records(SQL):
    print(day, genres, gross)

Output

2026-06-28 6 6629.86
2026-06-29 6 6327.10
2026-06-30 6 6670.05

The same code works wherever the connection comes from; BaseHook.get_connection("booknest_pg") returns the raw fields when no hook fits. Two CLI surprises: airflow connections list printed No data found, because it lists only the metadata database, and airflow connections get booknest_pg -o yaml printed password: booknest in clear text. Anyone who can run the CLI, or a task, can read every credential (Securing Orchestration).

MENU