Databricks 2,717 , founded by Spark 129 's creators, runs a managed Spark platform with its own runtime, Delta Lake 237,929 , the Unity Catalog governance layer and serverless options on AWS 24 , Azure 6 and Google Cloud 1 . It bills in Databricks Units (DBUs), a normalized unit of processing per hour, metered per second, at a rate that depends on the product: its pricing page lists Data Engineering (Lakeflow Jobs and pipelines) from $0.15 per DBU, Data Warehousing from $0.22 and interactive workloads from $0.40. How many DBUs an hour of compute consumes depends on the instance type, and on classic (non-serverless) compute your cloud provider bills the virtual machines separately; committed-use contracts lower the rates. Much of what Databricks adds is open source (Delta Lake, the Unity Catalog OSS project, MLflow), but its Photon engine is not: a vectorized C++ engine that replaces Spark's JVM execution for SQL and DataFrame work without code changes, on by default for serverless, SQL warehouses and jobs compute, and billed at a different DBU rate than the same instance without it. Benchmarks you run on open-source Spark therefore understate what the same code does there, in both speed and cost.
The former Community Edition has become Databricks Free Edition: serverless compute only, one workspace, one 2X-Small SQL warehouse, at most five concurrent job tasks, a fair-use quota that shuts compute down for the day when exceeded, and, by its terms, no commercial use. It suits learning this chapter's APIs, not running BookNest's production jobs.