ISTQB Certified Tester - AI Testing (ISTQB CT-AI) v2.0
In this certification course, you will gain a fundamental understanding of Artificial Intelligence, how to test and quality assure AI-based solutions, and how machine learning systems can be tested throughout the entire development lifecycle.
The course has been updated to ISTQB® Certified Tester AI Testing v2.0, which replaces the previous CT-AI v1.0 syllabus. The new syllabus has a clearer focus on testing AI-based systems, particularly machine learning, and now also includes key concepts and testing approaches within generative AI and large language models.
Prerequisites
- Knowledge and understanding of programming language – Java/Python/R
- Knowledge and understanding of statistics
- Experience with software development and testing
Target audience
- You will be able to contribute to the test strategy for AI-based systems and machine learning systems
- You will be able to contribute to the design and execution of test cases for AI-based systems
- You will gain an understanding of challenges related to testing AI-based systems, such as non-deterministic behavior, data dependency, probabilistic results, bias, transparency, robustness, and models that evolve over time
- You will gain an understanding of generative AI and testing large language models, including exploratory testing and red teaming
- You will gain an understanding of input data testing, including data quality, data representativeness, bias, data pipelines, and dataset validation
- You will learn about testing machine learning models, including ML performance metrics, adversarial testing, metamorphic testing, drift testing, A/B testing, and back-to-back testing
- You will gain an understanding of testing in relation to the development, deployment, and operation of machine learning systems
Content
- Introduction to Artificial Intelligence
Including the difference between AI-based and conventional systems, narrow AI, general AI, super AI, generative AI, ML frameworks, hosting, hardware, and relevant standards and regulations including EU AI act. - Quality characteristics for AI-based systems
Including AI-specific quality characteristics, functional correctness, adaptability, transparency, robustness, controllability, safety, ethics, and acceptance criteria. - Machine Learning (ML)
Including supervised, unsupervised, and reinforcement learning, the ML workflow, data preparation, datasets, pretrained models, fine-tuning, retrieval-augmented generation, ML performance metrics, and neural networks. - Testing AI-based systems
Including challenges in testing AI, locked and adaptive AI systems, statistical testing, test oracles, risk-based testing, test levels, and testing generative AI and large language models. - Input Data Testing for Machine Learning Systems
Including data quality, data-related risks, bias, data representativeness, data pipeline testing, dataset validation, and label correctness. - Model Testing for Machine Learning Systems
Including ML model risks, performance testing, adversarial testing, metamorphic testing, overfitting, underfitting, drift testing, A/B testing, and back-to-back testing. - Machine Learning Development Testing
Including testing during development and deployment, as well as monitoring and maintaining model performance in production.
Format
Certification
The exam is an official ISTQB-exam, and it is taken online.
Questions: 40 multiple choice, and no assistance is permitted.
Duration: 1 hour
Language: English. If English is not your native language, you can receive an additional 15 minutes for the exam.
Format: You must answer at least 29 out of 44 points to pass the exam. K3 (apply) questions are worth 2 points; K1/K2 are worth 1 point.
Do you have any questions please contact
- Charlotte Heimann
- Seniorspecialist
- +45 72203147