Senior QA Engineer (AI-based)

Senior QA Engineer

Summary

We are seeking a detail-oriented Senior QA Engineer to keep the quality assurance efforts across our complex data flows and AI-driven products, specifically focusing on conversational agents. This role requires a blend of automation expertise and strong analytical skills to define quality standards, validate complex data outputs, and ensure the reliable and responsible behavior of our generative AI systems.

Key Responsibilities

• Expertise in quality assurance efforts for data pipelines, APIs, and conversational agents.


• Design and execute generative AI testing scenarios, including exploratory testing for chatbot interactions, intent recognition, tone, and edge-case behavior.


• Experience in LLM test designs, including prompt and grounding validation, golden sets, and non-deterministic assertions.


• Perform crucial safety and quality checks, such as testing for hallucination, bias, toxicity, and PII leakage.


• Validate structured and unstructured data outputs, ensuring consistency, accuracy, and compliance.


• Establish and drive real-time testing approaches, including streaming data validation and API monitoring.


• Collaborate with ML and NLP teams to define comprehensive evaluation metrics and criteria for agent performance.


• Integrate AI-driven tools (like Copilot) into the QA lifecycle to accelerate test design, documentation, and defect analysis. 
  
  
  

Requirements

• 5+ years of experience in QA engineering, with proven experience in testing data systems or AI applications.


• Strong analytical skills and attention to detail, capable of spotting subtle inconsistencies in data and agent behavior.


• Proficiency in Python and SQL for data validation, automation scripting, and test preparation.


• Experience with API testing (REST/GraphQL automation and validation).


• Familiarity with chatbot frameworks, LLMs, or conversational testing.


• (Desirable) Experience or strong interest in using LLM-based assistants (e.g., GitHub Copilot, ChatGPT) in test and execution.


• (Desirable) Exposure to QA automation tools (e.g., PyTest, Postman, dbt tests). 

Location: Candidates based in UAE (Dubai or Abu Dhabi)

Come and join our #ParserCommunity. 

Follow us on Linkedin