Pipeline Performance

Pipeline Performance, Memory Limits, and Views

Three hard limits shape every pipeline. Each document the pipeline returns must fit in 16 MiB, though intermediate documents may exceed it; a pipeline may hold at most 1,000 stages; and each blocking stage gets 100 MB of RAM before it spills to disk. Since MongoDB 6.0 1,815 allowDiskUseByDefault is true, so spilling is automatic — db.adminCommand({ getParameter: 1, allowDiskUseByDefault: 1 }) confirms it on an 8.3 server. Pass { allowDiskUse: false } to one aggregate() call when you would rather see QueryExceededMemoryLimitNoDiskUseAllowed than let a report grind the disk.

Four habits keep pipelines clear of that territory: $match first on indexed fields, $limit right after any unindexed $sort, $unset of large arrays before a blocking stage, and $lookup last on an indexed foreign key. Verify each with explain('executionStats'): when totalDocsExamined far exceeds nReturned, the leading filter is not using an index.

Views

A view is a saved pipeline that behaves like a read-only collection. Every find() or aggregate() against it runs your pipeline first with the caller's stages appended, so total is computed on demand and never stored.

A view that derives order totals

db.createView('orderTotals', 'orders', [
  { $match: { status: { $ne: 'canceled' } } },
  { $set: { total: { $sum: { $map: { input: '$items', as: 'i',
                             in: { $multiply: ['$$i.qty', '$$i.price'] } } } } } },
  { $project: { customerId: 1, placedAt: 1, status: 1, total: 1 } }
])
db.orderTotals.aggregate([
  { $group: { _id: '$status', revenue: { $sum: '$total' } } }, { $sort: { _id: 1 } }
])

Output

[
  { _id: 'delivered', revenue: 3080 },
  { _id: 'pending', revenue: 150 },
  { _id: 'shipped', revenue: 775 }
]

Views cost nothing to store and cannot drift from the source, so they are the right way to hand an application a curated shape, or to hide fields from a role allowed to read orderTotals but not orders. A view is read-only (insertOne fails with Namespace shop.orderTotals is a view, not a collection) and carries no indexes of its own, using the source collection's. When the pipeline is too expensive per request, build an on-demand materialized view: the same pipeline ending in $merge ($out and $merge), refreshed on a schedule. You trade freshness for speed and gain a collection you can index.

MENU

Views