This practical course provides an accessible starting point for professionals who want to learn Apache Spark programming using Databricks and develop skills for large-scale data processing.

The programme consists of four modules of approximately four hours each. Participants begin with Apache Spark's distributed architecture before progressing to data processing with the DataFrame API, ETL development, real-time processing with Structured Streaming, and the optimisation of Spark workloads on Databricks.

Throughout the course, Apache Spark, Python, the Spark DataFrame API, Structured Streaming, Delta Lake, and Databricks are brought together in practical data engineering scenarios. Core concepts are reinforced through demonstrations and hands-on labs, allowing participants to apply their knowledge to realistic data processing requirements.

The later stages of the programme explore production-oriented topics including Lakehouse architecture, Delta Lake, Spark performance optimisation, and the monitoring of Spark workloads within Databricks.

We can organize this training at your preferred date and location.

Prerequisites

Participants should have:

Familiarity with Python programming
An understanding of data types, lists, dictionaries, and variables
Experience with functions, loops, and conditional statements
Knowledge of exception handling
Familiarity with accessing classes and using third-party Python libraries
Basic SQL knowledge
The ability to write queries using SELECT, WHERE, GROUP BY, ORDER BY, LIMIT, and JOIN

Who Should Attend

This course is particularly suitable for:

Data engineers and data scientists who want to strengthen their Apache Spark programming skills
Developers looking to use Apache Spark and Delta Lake within Databricks
Professionals involved in large-scale data processing and real-time analytics

What You Will Learn

Introduction to Apache Spark

Participants develop an understanding of Apache Spark's core concepts and distributed architecture, with a focus on its practical role in large-scale data processing.

The module introduces Spark programming frameworks and the DataFrame API while developing the skills required to read, write, and transform data through Python-based Spark workflows.

Developing Applications with Apache Spark

Participants learn how to build scalable data processing applications with Apache Spark. Using the Spark DataFrame API, they develop ETL pipelines and work with distributed data transformations.

The course covers grouping, aggregation, joins, set operations, and window functions together with complex data types such as arrays, maps, and structs. Participants also explore good practices for improving Spark application performance.

Stream Processing and Analysis with Apache Spark

This part of the course introduces the principles of processing real-time and continuously arriving data.

Participants develop applications with the Spark Structured Streaming API and explore techniques including stream aggregation and window analysis. The module develops the practical skills required to create scalable and fault-tolerant streaming applications for dynamic data environments.

Monitoring and Optimising Apache Spark Workloads on Databricks

This module explores scalable data workflows through Lakehouse Architecture and Medallion Architecture concepts.

Participants examine data governance, access control, and lineage with Unity Catalog while exploring how Delta Lake can support reliable, ACID-compliant data pipelines.

Spark optimisation methods including partitioning, caching, and query tuning are also considered alongside performance monitoring, troubleshooting, and good practices for efficient data engineering and analytics workloads.

Training Outline

Module 1: Introduction to Apache Spark

Spark Runtime Architecture
Exploring Apache Spark architecture in Databricks
Introduction to Spark DataFrames and SQL
Reading and writing data with DataFrames
Distributed system programming fundamentals
Basic ETL with the DataFrame API
Flight data ETL using the DataFrame API
Analysing transaction data with DataFrames

Module 2: Developing Applications with Apache Spark

DataFrame API basics
Demo (Optional): Basic ETL with the DataFrame API
Grouping and aggregating data
Demo: Grouping and aggregating data
Lab: Grouping and aggregating e-commerce data
Relational operations
Demo: Data relational operations in Apache Spark
Working with complex data
Demo: Working with complex data types in Apache Spark
Lab: Working with complex data types in e-commerce data

Module 3: Stream Processing and Analysis with Apache Spark

Introduction to stream processing
Spark Structured Streaming
Demo: Introduction to Spark Structured Streaming
Lab: Introduction to Spark Structured Streaming
Advanced stream processing and analysis
Demo: Window aggregation in Spark Structured Streaming
Lab: Window aggregation in Spark Structured Streaming

Module 4: Monitoring and Optimising Apache Spark Workloads on Databricks

Apache Spark and Databricks
Using Apache Spark with Delta Lake
Demo: Introduction to Delta Lake
Lab: Introduction to Delta Lake
Optimising Apache Spark
Demo: Optimising Apache Spark
Lab: Optimising Apache Spark

Why Choose Us

Experience Apache Spark Programming with Databricks through Bilginç IT Academy's live and interactive virtual classroom environment. Join a Public course as an individual delegate or arrange a dedicated Private / In-house online training program exclusively for your organization.

Delivery Method: Online Instructor-Led
Participation Model: Public / Private (In-house)
Live and Interactive Training: Connect with your instructor in real time and actively participate through discussions, Q&A sessions, practical exercises, and group activities.
Flexible Participation: Join the training from your home, office, or any location with a suitable internet connection.
Expert Trainer Network: Learn from experienced trainers with strong industry backgrounds and practical field expertise.
Over 30 Years of Training Expertise: Benefit from Bilginç IT Academy's professional training experience since 1995.
Worldwide Access: Join our live virtual classrooms from anywhere in the world or arrange a dedicated online training program for your organization.

Experience Apache Spark Programming with Databricks in a professional face-to-face classroom environment. Classroom Based training can be delivered as a Public course open to individual delegates or as a dedicated Private / In-house class exclusively for your organization.

Delivery Method: Classroom Based
Participation Model: Public / Private (In-house)
Face-to-Face Learning: Interact directly with your instructor and fellow delegates in an engaging classroom environment.
Experienced Trainers: Learn from specialists with extensive industry experience and practical real-world knowledge.
Professional Training Environment: Attend training in comfortable, well-equipped classrooms designed to support effective learning.
Practical Learning: Depending on the course, reinforce your knowledge through hands-on exercises, scenarios, case studies, and instructor-led activities.

Arrange Apache Spark Programming with Databricks as a dedicated Onsite training program for your organization. Bilginç IT Academy trainers can deliver the training at your office or another location of your choice, with the program planned around your team's requirements and business objectives.

Delivery Method: Onsite
Participation Model: Private (In-house)
Training at Your Preferred Location: Organize the training at your company's office or another location selected by your organization.
Tailored Course Content: Adapt the training program to your projects, team structure, existing skill levels, and specific business requirements.
Team-Focused Learning: Develop your team around a shared knowledge base while strengthening internal collaboration and knowledge transfer.
Flexible Scheduling: Plan the training dates, location, and program according to your organization's operational requirements.
Worldwide Onsite Delivery: Bilginç IT Academy trainers can travel internationally to deliver dedicated training programs at your preferred location.

Why have you chosen us?

I have attended a training from Bilginc IT Academy before and I was satisfied.

I have attended a training from a different provider and it was not helpful.

Other

How many employees do you have in your IT department?

0 – 50

50 – 250

250 – 1000

1000+

Apache Spark Programming with Databricks Training