DataFrames and Laziness

Spark Under the Hood showed what the engine does with a DataFrame; this section is about writing them: reading and writing BookNest's files (the formats themselves are JSON, Columnar and Binary Formats's subject), fixing schemas, and seeing why nothing runs until you ask for a result.

Subsections

• Creating a SparkSession
• Reading Files
• Schemas
• Writing Files
• Save Modes and Write Options
• Transformations Versus Actions
• Lazy Evaluation
• Column Expressions
• Loading Order History