MyPhatshala

Apache Spark and PySpark

The fastest engine for big data processing, used through Python.

Intermediate6 modules24 lessonsFree · no sign-up
0 of 24 lessons published
View syllabus

Syllabus

01 IntroductionWhy Spark? 4 lessons
  1. 1.1What is Apache Spark?Coming soon
  2. 1.2Spark architecture: driver, executors, cluster managerComing soon
  3. 1.3How a Spark job runs: jobs, stages and tasksComing soon
  4. 1.4Installing PySparkComing soon
02 RDDsSpark's core abstraction. 4 lessons
  1. 2.1What is an RDD?Coming soon
  2. 2.2Transformations and actionsComing soon
  3. 2.3Lazy evaluation and the DAGComing soon
  4. 2.4Narrow vs wide transformationsComing soon
03 DataFramesStructured data in Spark. 7 lessons
  1. 3.1DataFrames and SparkSessionComing soon
  2. 3.2Reading and writing dataComing soon
  3. 3.3Selecting, filtering and adding columnsComing soon
  4. 3.4GroupBy and aggregationsComing soon
  5. 3.5JoinsComing soon
  6. 3.6Window functionsComing soon
  7. 3.7UDFsComing soon
04 Spark SQLSQL on big data. 2 lessons
  1. 4.1Spark SQLComing soon
  2. 4.2Catalyst optimiserComing soon
05 PerformanceMaking jobs fast. 4 lessons
  1. 5.1Partitioning and shufflingComing soon
  2. 5.2Caching and persistenceComing soon
  3. 5.3Broadcast variables and accumulatorsComing soon
  4. 5.4Tuning Spark jobsComing soon
06 Streaming and MLBeyond batch. 3 lessons
  1. 6.1Structured streamingComing soon
  2. 6.2Spark MLlibComing soon
  3. 6.3Spark on DatabricksComing soon