Start from the business need, not the technology. Ask how quickly someone must act on the data, not how quickly they would like to see it. A finance report read once a morning gains nothing from second-level freshness; a fraud check that fires after the parcel has shipped is worthless. A good rule of thumb: use streaming when the pipeline needs real-time or near-real-time event ingestion and several downstream consumers, and batch when latency is not critical.
| Question | Points to batch | Points to streaming |
|---|---|---|
| How fresh must results be? | Hours or a day | Seconds or minutes |
| What happens with the result? | Reports, history, training data | Alerts, live decisions, user features |
| How many consumers per event? | One or two | Many independent consumers |
| Team experience | SQL and scheduled jobs | Running always-on services |
| Cost tolerance | Pay per run | Pay for capacity around the clock |
The choice is not all or nothing. Many teams ingest as a stream and process in batches: events flow into Kafka 129 and land continuously in lakehouse tables, and most transformations still run every hour or night over those tables (Streaming into the Lakehouse). An hourly batch is often "real-time enough" at a fraction of the operational effort. At BookNest, the daily sales mart is batch, while the live order counter and low-stock alerts are streaming. The next section looks at architectures that combine the two.