AI Verification Engineer (m/f/d)

As an AI Verification Engineer, you will play a critical role in ensuring the quality, reliability, and trustworthiness of our AI models.
You will be responsible for designing, developing, and executing comprehensive verification strategies to validate the performance, robustness, fairness, and safety of our AI systems. This role requires a deep understanding of AI model architectures, machine learning principles, and rigorous testing methodologies. You will work closely with AI experts, MLOps engineers, and product teams to integrate verification processes throughout the AI development lifecycle.
 
Your main tasks and responsibilities include:

• AI Model Verification Strategy: Design and implement comprehensive verification plans for various AI models (e.g., deep learning–based object detection), including the definition of test objectives, metrics, and success criteria
• Test Case Development: Develop and automate a wide range of test cases—including functional, performance, adversarial, robustness, fairness, and safety tests—for validating AI models
• Data Selection for Verification: Strategically select, curate, and augment datasets specifically for verification purposes. Identify edge cases, corner cases, and potential biases to ensure thorough model evaluation
• Tooling and Infrastructure Development: Build and maintain scalable infrastructure and tools to automate testing, reporting, and performance analysis of AI models
• Performance and Robustness Analysis: Analyse model behaviour under varying conditions, identify failure modes, and quantify robustness against perturbations and adversarial attacks
• Bias and Fairness Assessment: Develop and apply methodologies to detect and mitigate biases in AI models, ensuring fair and equitable outcomes across different demographic groups
• Safety and Ethical AI Verification: Design tests to validate that AI models adhere to safety guidelines and ethical principles, identifying potential risks and unintended consequences
• Results Analysis and Reporting: Interpret complex verification results, prepare clear and concise reports, and effectively communicate findings to the AI team and other stakeholders
• Collaboration: Work closely with the AI team and product owners to provide feedback on model design and highlight areas for improvement. Collaborate with MLOps to integrate verification into CI/CD pipelines
• Documentation: Maintain comprehensive documentation of verification processes, test plans, methodologies, and outcomes
• Stay Current: Keep up to date with the latest developments in AI, machine learning, and AI verification techniques and tools

• Bachelor’s or Master’s degree in computer science, Electrical Engineering, Machine Learning, Robotics, or a related quantitative field
• 3+ years of experience in software quality assurance, verification, or validation, including at least 1 year specifically focused on AI/ML model verification
• Strong understanding of machine learning algorithms and deep learning architectures (e.g., CNNs for object detection), along with their underlying principles
• Proficiency in programming languages commonly used in AI development, particularly Python
• Hands-on experience with machine learning frameworks such as TensorFlow or PyTorch
• Experience with data manipulation and analysis libraries (e.g., Pandas, NumPy)
• Solid understanding of statistical methods and data analysis techniques for evaluating model performance
• Familiarity with common AI fairness, explainability, and safety frameworks and techniques
• Experience with version control systems (e.g., Git)
• Excellent problem-solving skills and strong attention to detail
• Strong communication and collaboration abilities
• English proficiency at C1 level or higher 
• Basic German skills (minimum B1 level) are required
• Current residence in Germany

 
Preferred Qualifications:

• Experience with MLOps practices and tools (e.g., DVC, CI/CD)
• Familiarity with cloud platforms (e.g., AWS) for handling large volumes of test data
• Experience with synthetic data–based frameworks (e.g., Blender, CARLA, Unreal Engine)
• Knowledge of formal verification methods for AI systems