D

Senior QA Engineer Agentic AI Testing & Evaluation

Job Description - Senior QA Engineer Agentic AI Testing & Evaluation

About the Role

Dailoqa is looking for an experienced Senior QA Engineer
– Agentic AI Testing & Evaluation
to own the quality, testing, and
evaluation strategy for our Broccoli Platform. This role will be
responsible for establishing robust testing frameworks for AI agents,
multi-agent workflows, orchestration layers, tools, and enterprise-grade AI
applications.

The ideal candidate will have deep expertise in test
automation, quality engineering, AI/ML validation, security testing, and
performance engineering
, with a strong understanding of Large Language
Models (LLMs), Agentic AI systems, and modern DevOps practices.

You will work closely with Capability Leads, AI
Architects, Product Teams, and Platform Engineers
to ensure that Broccoli
delivers reliable, secure, scalable, and high-performing AI solutions.



Requirements

Required Skills & Qualifications

Technical Expertise

  • 7–10
    years of experience in Quality Assurance, Test Automation, or Quality
    Engineering.
  • Strong
    experience with automation frameworks using Python, Java, or
    JavaScript/TypeScript
    .
  • Hands-on
    experience with:
    • Selenium,
      Playwright, Cypress, or Robot Framework
    • API
      testing tools such as Postman, Rest Assured, or Karate
    • CI/CD
      platforms including Azure DevOps, GitHub Actions, Jenkins, or GitLab
  • Experience
    with cloud platforms such as Azure, AWS, or Google Cloud.

Agentic AI & LLM Testing

  • Understanding
    of:
    • Large
      Language Models (LLMs)
    • Retrieval-Augmented
      Generation (RAG)
    • AI
      Agents and Agent Orchestration Frameworks
    • Prompt
      Engineering and Evaluation
  • Experience
    with AI evaluation platforms and frameworks is preferred.
  • Knowledge
    of hallucination detection, safety validation, guardrails, and model
    benchmarking.

Performance & Security Testing

  • Experience
    with performance testing tools such as JMeter, k6, or Locust.
  • Knowledge
    of application security testing and threat modeling.
  • Familiarity
    with AI security testing, adversarial testing, and governance controls.

Soft Skills

  • Strong
    analytical and problem-solving skills.
  • Excellent
    stakeholder management and communication abilities.
  • Experience
    working in cross-functional Agile teams.
  • Ability
    to lead quality initiatives in fast-paced, innovation-driven environments.

Preferred Qualifications

  • Experience
    testing AI-powered enterprise applications.
  • Exposure
    to Microsoft Azure AI, OpenAI, LangChain, LangGraph, CrewAI, Semantic
    Kernel, or similar agent frameworks.
  • Certifications
    such as:
    • ISTQB
      Advanced Test Analyst/Test Manager
    • Certified
      Agile Tester
    • Azure
      AI Engineer Associate
    • Azure
      DevOps Engineer Expert
  • Experience
    working with BFSI, enterprise transformation, or regulated industry
    environments.


Original job Senior QA Engineer Agentic AI Testing & Evaluation posted on GrabJobs ©. To flag any issues with this job please use the Report Job button on GrabJobs.
Share Job
Share Job

Similar Senior QA Engineer Jobs in India

GrabJobs is the no1 job portal in India, connecting you to thousands of jobs fast! Find the best jobs in India, apply in 1 click and get a job today!

Copyright © 2026 Grabjobs Pte.Ltd. All Rights Reserved.