Why Columns Win Wide Scans

Why Columnar Layout Wins at Wide Scans

Analytical tables are wide; analytical queries are narrow. demos/ch04/py/wide_scan.py times gross sales per genre (two columns) on sales and on sales_wide, which adds customer name, email and book summary. Each engine gets one core, the tables are cached, and the best of seven runs counts.

Gross sales per genre on one core, best of seven runs (shared 4-CPU host)
Table PostgreSQL 1,289 heap PostgreSQL time DuckDB 61,228 time
sales (15 columns) 19.9 MB 39.9 ms 1.9 ms
sales_wide (18 columns) 34.6 MB 44.6 ms 1.8 ms

Widening the table grew PostgreSQL's heap 74 percent and slowed its scan; DuckDB never opened the new columns. With everything cached the gap is not disk I/O: DuckDB was 20 to 75 times faster across five runs (higher while other lanes loaded the host) because it decodes two compressed columns in vectors (The Vectorized Execution Model) instead of deforming 137,944 tuples. On data larger than memory, reading 2 of 18 columns widens the gap further (Why Analytics Favors Columns).