Partitioning and Caching

Much of Spark 129 tuning comes down to how data is split, in memory and on disk, and whether a result is computed once or every time it is used.

Subsections

• Memory vs Disk Partitions
• Repartition and Coalesce
• Partitioned Table Writes
• Bucketing for Repeated Joins
• Caching and Persistence Levels
• Choosing a Storage Level
• Caching Strategy