What This Chapter Runs

What This Chapter Runs For Real and What It Documents

Everything open source here runs on the 4-CPU WSL2 6 host in Docker 514 containers prefixed l1- on one network, l1-lake, one heavy service at a time; timings come from a shared machine, so compare ratios.

The stack: MinIO 30,943 RELEASE.2025-10-15T17-29-55Z as object storage (container l1-minio); Apache Iceberg 129 1.12.0 tables behind an Iceberg REST catalog (l1-iceberg-rest) and Delta Lake 237,929 through delta-rs 1.6.6; Spark 129 4.1.3, Trino 403,499 483 and DuckDB 61,228 1.5.5 as engines; and the Iceberg sink for Kafka Connect 129 , GX Core, Soda Core 2,433 and OpenLineage for streaming, quality and lineage.

Two version choices need explaining. MinIO's community edition no longer ships binaries or images, so Object Storage and MinIO builds its last release from source. Iceberg 1.12.0, released on 30 September 2026, ships Spark runtime jars for Spark 3.5, 4.0 and 4.1 only; the 4.1 jar fails on Batch Processing with Apache Spark's Spark 4.2.0 at the first catalog call (IncompatibleClassChangeError on SparkView), so this chapter runs Spark 4.1.3 beside it.

The managed lakehouses (Databricks 2,717 , Snowflake with Iceberg tables, AWS 24 S3 Tables with Glue and Lake Formation, BigQuery 1 with BigLake, and Microsoft Fabric 6 with OneLake) offer no usable tier without a credit card. Managed Lakehouse Platforms describes them from official architecture and pricing pages, labeled "not run here".