Programming

4 Tips To Write Scalable Apache Spark Code

Introduction

In this article, I will share some tips on how to write scalable Apache Spark code. The examples presented here are actually based on the code I encountered in the real world. So, by sharing these tips, I hope I can help newcomers to write performant Spark code without needlessly increasing their cluster’s resources.

Cluster Setup

The cluster I used to run the code in this article is hosted on Databricks with the following…