Role
Overview 

As a Test
Automation Lead at Dailoqa, you’ll architect and implement robust testing
frameworks for both software and AI/ML systems. You’ll bridge the gap between traditional
QA and AI-specific validation, ensuring seamless integration of automated
testing into CI/CD pipelines while addressing unique challenges like model
accuracy, GenAI output validation, and ethical AI compliance. 

Key
Responsibilities 

Test
Automation Strategy & Framework Design 

• Design and implement scalable
  test automation frameworks for frontend (UI/UX), backend APIs, and AI/ML model-serving endpoints using tools like Selenium,
  Playwright, Postman, or custom Python/Java solutions. 
  

• Build GenAI-specific test suites for validating prompt outputs,
  LLM-based chat interfaces, RAG systems, and vector search accuracy. 
  

• Develop performance testing strategies for AI pipelines (e.g., model
  inference latency, resource utilization). 
  

Continuous
Testing & CI/CD Integration 

• Establish and maintain continuous testing pipelines integrated with GitHub Actions,
  Jenkins, or GitLab CI/CD. 
  

• Implement shift-left testing by embedding automated checks
  into development workflows (e.g., unit tests, contract testing). 
  

AI/ML
Model Validation 

• Collaborate with data
  scientists to test AI/ML models for accuracy, fairness, stability, and bias mitigation using tools like TensorFlow
  Model Analysis or MLflow. 
  

• Validate model drift and retraining pipelines to
  ensure consistent performance in production. 
  

Quality
Metrics & Reporting 

• Define and track KPIs.
  

• Test coverage (code, data, scenarios) 
  

• Defect leakage rate 
  

• Automation ROI (time saved vs. maintenance
  effort) 
  

• Model accuracy thresholds 
  

• Report risks and quality trends
  to stakeholders in sprint reviews. 
  

• Drive adoption of AI-specific
  testing tools (e.g., LangChain for LLM testing, Great Expectations for
  data validation). 
  

 Soft
Skills 

• Strong problem-solving skills
  for balancing speed and quality in fast-paced AI development. 
  

• Ability to communicate
  technical risks to non-technical stakeholders. 
  

• Collaborative mindset to work
  with cross-functional teams (data scientists, ML engineers, DevOps). 
  

Requirements

Technical
Requirements 

Must-Have 

• 10 years in test automation,
  with 2+ years validating AI/ML systems. 
  
• Expertise in: Automation tools: Selenium, Playwright,
  Cypress, REST Assured, Locust/JMeter 
  
• CI/CD: Jenkins, GitHub Actions,
  GitLab 
  

• AI/ML testing: Model validation, drift
  detection, GenAI output evaluation 
  

• Languages: Python, Java, or JavaScript 
  

• Certifications: ISTQB Advanced,
  CAST, or equivalent. 
  

• Experience with MLOps tools: MLflow, Kubeflow, TFX 
  

• Familiarity with vector databases (Pinecone, Milvus) and RAG workflows. 
  

• Strong programming/scripting
  experience in JavaScript, Python, Java, or similar 
  

• Experience with API testing, UI
  testing, and automated pipelines 
  

• Understanding of AI/ML model testing, output evaluation, and non-deterministic
  behavior validation 
  

• Experience with testing AI chatbots, LL​M responses, prompt engineering
  outcomes, or AI fairness/bias 
  

• Familiarity with MLOps pipelines and automated validation of model performance
  in production 
  

• Exposure
  to Agile/Scrum methodology and tools like Azure Boards