DataFrame of Mind

Overview

Build data pipelines with Polars and PySpark. Learn when to use single-node processing and when to scale out.

What you'll cover

• ▸Operational vs. analytical data and schema strategies
• ▸Relational algebra: project, filter, join, aggregate, window
• ▸Polars expressions, contexts, and lazy evaluation
• ▸PySpark DataFrame API and distributed processing
• ▸Query optimization and user-defined functions
• ▸The big data landscape: single-node vs. distributed

What you'll be able to do

• ✓Build data transformation pipelines in Polars and PySpark
• ✓Apply relational operations across both frameworks
• ✓Use lazy evaluation and query plans to optimize performance
• ✓Choose the right processing tool for a given workload