4 years ago

Category: Deployment, Docker

While the Scala Spark API plays a significant role in the data lake ELT workloads and data engineering, the data science and advanced analytics spaces are almost exclusively PySpark workloads written in Python. And most of these Spark workloads get executed on AWS EMR clusters.