Explore Azure Databricks
- Overview of Azure Databricks as a cloud-scale platform for data analytics and machine learning
- Key concepts and workloads in Azure Databricks
- Data governance using Unity Catalog and Microsoft Purview
- Hands-on exercise: Exploring Azure Databricks
Use Apache Spark in Azure Databricks
- Introduction to Apache Spark and its role in large-scale data analytics
- Creating and managing Spark clusters
- Using Spark notebooks to process and transform large datasets
- Working with structured and unstructured data files in Spark
- Visualising data using Spark and Databricks
- Hands-on exercise: Using Spark for data analytics
Train a machine learning model in Azure Databricks
- Principles of machine learning and predictive modelling
- Preparing data for machine learning, including feature engineering and transformations
- Training machine learning models using Scikit-Learn, PyTorch, and TensorFlow
- Evaluating model performance using standard machine learning metrics
- Hands-on exercise: Training a machine learning model
Use MLflow in Azure Databricks
- Introduction to MLflow and its role in the machine learning lifecycle
- Running experiments and tracking performance metrics with MLflow
- Registering, serving, and managing models using MLflow
- Deploying trained models for inference within Databricks
- Hands-on exercise: Using MLflow for model management
Tune hyperparameters in Azure Databricks
- Understanding hyperparameter tuning and its impact on machine learning models
- Using Hyperopt for automated hyperparameter tuning in Azure Databricks
- Reviewing and analysing Hyperopt trials for optimisation insights
- Scaling Hyperopt trials for improved performance
- Hands-on exercise: Tuning model hyperparameters using Hyperopt
Use AutoML in Azure Databricks
- Overview of AutoML and its benefits in machine learning
- Running AutoML experiments via the Azure Databricks user interface
- Using Python code to execute AutoML workflows
- Comparing AutoML results with traditional model development
- Hands-on exercise: Using AutoML for machine learning model development
Train deep learning models in Azure Databricks
- Fundamentals of deep learning and neural networks
- Training deep learning models using PyTorch in Databricks
- Using TorchDistributor for distributed deep learning model training
- Deploying deep learning models for real-world AI tasks
- Hands-on exercise: Training and optimising deep learning models in Databricks
Manage machine learning in production with Azure Databricks
- Automating data transformations and machine learning workflows in Databricks
- Exploring model development, versioning, and lifecycle management
- Deploying models for real-time inference and decision-making
- Monitoring deployed models for performance and drift detection
- Hands-on exercise: Managing a machine learning model in production
Exams and assessments
This course does not include formal exams. Participants will complete interactive labs and knowledge checks to reinforce learning outcomes.
Hands-on learning
This course includes:
- Hands-on labs for data processing, model training, hyperparameter tuning, and model deployment.
- Practical exercises using Apache Spark, MLflow, AutoML, and Hyperopt.
- Real-world case studies on implementing scalable machine learning solutions.
