Chapter 1: Introduction to AI
- Definition of AI and AI Effect
- Narrow, General and Super AI
- AI-based and Conventional Systems
- AI Technologies
- AI Development Frameworks
- Hardware for AI-Based Systems
- AI as a Service (AIaaS)
- Pre-Trained Models
- Standards, Regulations and AI
Chapter 2: Quality Characteristics for AI-Based Systems
- Flexibility and Adaptability
- Autonomy
- Evolution
- Bias
- Ethics
- Side Effects and Reward Hacking
- Transparency, Interpretability and Explainability
- Safety and AI
Chapter 3: Machine Learning (ML) – Overview
- Forms of ML
- ML Workflow
- Selecting a Form of ML
- Factors Involved in ML Algorithm Selection
- Overfitting and Underfitting
Chapter 4: ML – Data
- Data Preparation as Part of the ML Workflow
- Training, Validation and Test Datasets in the ML Workflow
- Dataset Quality Issues
- Data Quality and its Effect on the ML Model
- Data Labelling for Supervised Learning
Chapter 5: ML Functional Performance Metrics
- Confusion Matrix
- Additional ML Functional Performance Metrics for Classification, Regression and Clustering
- Limitations of ML Functional Performance Metrics
- Selecting ML Functional Performance Metrics
- Benchmark Suites for ML Performance
Chapter 6: ML – Neural Networks and Testing
- Neural Networks
- Coverage Measures for Neural Networks
- Chapter 7: Testing AI-Based Systems Overview
- Specification of AI-Based Systems
- Test Levels for AI-Based Systems
- Test Data for Testing AI-Based Systems
- Testing for Automation Bias in AI-Based Systems
- Documenting an AI Component
- Testing for Concept Drift
- Selecting a Test Approach for an ML System
Chapter 8: Testing AI-Specific Quality Characteristics
- Challenges Testing Self-Learning Systems
- Testing Autonomous AI-Based Systems
- Testing for Algorithmic, Sample and Inappropriate Bias
- Challenges Testing Probabilistic and Non-Deterministic AI-Based Systems
- Challenges Testing Complex AI-based Systems
- Testing the Transparency, Interpretability and Explainability of AI-Based Systems
- Test Oracles for AI-Based Systems
- Test Objectives and Acceptance Criteria
Chapter 9: Methods and Techniques for the Testing of AI-Based Systems
- Adversarial Attacks and Data Poisoning
- Pairwise Testing
- Back-to-Back Testing
- A/B Testing
- Metamorphic Testing (MT)
- Experience-based testing of AI-based Systems
- Selecting Test Techniques for AI-based Systems
Chapter 10: Test Environments for AI-Based Systems
- Test Environments for AI-Based Systems
- Virtual Test Environments for Testing AI-Based Systems
Chapter 11: Using AI for Testing
- AI Technologies for Testing
- Using AI to Analyze Reported Defects
- Using AI for Test Case Generation
- Using AI for the Optimization of Regression Test Suites
- Using AI for Defect Prediction
- Using AI for Testing User Interfaces
Exams and Assessments
Your course fee includes an iSQI voucher for the examination which you will book at a later date.
The format of the exam is multiple choice.
- Exam duration is 60 minutes. If the candidate’s native language is not the examination language, the candidate is allowed an additional 25% (exam duration = 75 minutes).
- There are 40 questions.
- To pass the exam, at least 65% of the total sum of points must be answered correctly.
- The total number of points for this exam should be set at 47 points. Therefore, a minimum of 31 points is required to achieve a passing score.
Hands-On Learning
Hands-on Machine Learning Concepts:
Learners engage in exercises that illustrate key ML concepts such as overfitting and underfitting. Activities include creating simulated datasets, training simple models (like linear regression), and visualizing model performance under different data conditions (e.g., limited data, weak feature-target correlations). Participants analyze results using metrics like Mean Squared Error (MSE) and R², and interpret graphical outputs to understand model behavior.
Test Design and Reduction Techniques:
One exercise focuses on combinatorial test design. Learners are tasked with defining a model with multiple parameters (e.g., model type, number of estimators, training rate, etc.), generating a large set of possible parameter combinations, and then applying pairwise testing to reduce the number of test cases. This introduces practical skills in test optimization and the use of tools (such as Microsoft PICT) for efficient test coverage.
