Chunking and Cursors

Chunking, Lazy Streaming, and Cursors

get() builds every row at once: fine for a page, fatal for an export. chunk() pages with OFFSET, chunkById() with where id > last, lazy() and lazyById() wrap the same in a LazyCollection, and cursor() hydrates one row at a time. This command read a 200,000-row views table (filled by a recursive CTE, Recursive CTEs) each way:

Appended to routes/console.php: five ways to read 200,000 rowsPHP
Artisan::command('views:scan {how}', function (string $how) {
    [$views, $n, $t] = [DB::table('views'), 0, microtime(true)];
    $count = function () use (&$n) { $n++; };
    match ($how) {
        'get' => $views->get()->each($count),
        'cursor' => $views->cursor()->each($count),
        'lazy' => $views->orderBy('id')->lazy(1000)->each($count),
        'lazyById' => $views->lazyById(1000)->each($count),
        'chunkById' => $views->chunkById(1000, fn ($rows) => $rows->each($count)),
    };
    $this->line(sprintf('%-9s %d rows %.2f s %.1f MB', $how, $n, microtime(true) - $t,
        memory_get_peak_usage() / 1048576));
});
Output
get       200000 rows 0.26 s 126.7 MB
cursor    200000 rows 0.27 s 32.2 MB
lazy      200000 rows 4.05 s 25.5 MB
lazyById  200000 rows 0.48 s 25.5 MB
chunkById 200000 rows 0.51 s 25.0 MB

Each line is a separate php artisan views:scan <how> process. get() peaked 100 MB above the paging methods; cursor() less, since PDO's MySQL 524 driver buffers the whole result by default. lazy() was eight times slower, each OFFSET query re-reading the rows before it (Keyset Pagination). Prefer the ById variants, above all when the loop updates the filtered column: a chunk() loop zeroing stock left 3 of 7 in-stock products untouched.