httpfs reads files over HTTP(S) and reads and writes them through the S3 API that Amazon S3 24 , Google Cloud 1 Storage, Cloudflare 2 R2 and MinIO 30,943 speak, fetching byte ranges rather than whole files. httpfs_demo.sh serves files/ from an nginx 75 container that logs each request's Range header, and queries June's orders over HTTP:
W=/home/dev/v7-l2/ch04
cp "/mnt/d/Books/Data Engineering/demos/ch04/http/nginx.conf" $W/nginx.conf
docker rm -f l2-nginx >/dev/null 2>&1
docker run -d --name l2-nginx -p 32080:80 -v $W/files:/usr/share/nginx/html:ro \
-v $W/nginx.conf:/etc/nginx/conf.d/default.conf:ro nginx:1.30-alpine >/dev/null
for i in $(seq 30); do curl -s -o /dev/null localhost:32080/ && break; sleep 1; done
cd $W && /home/dev/v7-l2/tools/duckdb-1.5.5/duckdb -c "SET TimeZone = 'UTC'" -c "
SELECT count(*) AS orders, sum(total) AS revenue
FROM 'http://localhost:32080/orders.parquet' WHERE order_ts >= '2026-06-01'"
docker logs l2-nginx 2>/dev/null | grep orders.parquet
docker rm -f l2-nginx >/dev/null... │ 5498 │ 187963.78 │ └────────┴───────────────┘ HEAD /orders.parquet "-" 200 0 GET /orders.parquet "bytes=782330-798713" 206 16384 GET /orders.parquet "bytes=640733-704715" 206 63983 GET /orders.parquet "bytes=764128-782329" 206 18202
DuckDB 61,228 asked for the size, read the last 16 kB (footer and metadata), then fetched only two column chunks of the last row group: 98,569 bytes of 798,714 in 206 Partial Content replies. Against object storage each file costs a few requests, so latency dominates: prefer fewer, larger files. For S3 or MinIO, store credentials in a secret (CREATE SECRET (TYPE s3, KEY_ID '...', SECRET '...', REGION '...', ENDPOINT '...')) and query s3://bucket/path/*.parquet with globs and Hive 129 partitions as in Globs and Hive Partitioning; Object Storage and MinIO sets up MinIO.