AI System Quality Assurance Engineer Data & Agentic AI

Role– AI System Quality Assurance Engineer –
Data & Agentic AI 

Role Summary - This role will focus on
validating data movement, mapping, transformation, and integrity across
migration testing and the integration between our agentic AI quoting platform
and SubmissionLink. AI agents consume Small Business Owner information
provided through SubmissionLink and backed by structured data models
to create insurance applications.

The role will verify what data is being used, how it moves
through the system, and whether it is correctly validated at each
stage, from source payloads and migration outputs through APIs, AI agents,
business rules, guardrails, and the user interface. It requires strong API,
integration, and data-validation skills, along with practical exposure to
agentic testing and validation of AI-generated outputs.

The candidate must have strong communication skills
and be capable of working directly with US-based Product, Data Intelligence,
Engineering, and business stakeholders.

Key Responsibilities -

• Understand
  what data is being used and map how it moves through each stage of the
  system pipeline 
  

• Perform
  data validation during migration testing, including source-to-target
  comparison, completeness checks, accuracy checks, and transformation
  validation 
  

• Validate SubmissionLink data
  against expected Small Business Owner information and downstream
  insurance-application outputs 
  

• Verify
  that AI agents correctly consume, interpret, and
  apply SubmissionLink data when creating insurance
  applications 
  

• Validate
  data mapping and transformation across source systems, APIs, AI agents,
  business rules, guardrails, and the UI 
  

• Build
  or execute validation scripts to check data integrity, completeness,
  schema conformance, and transformation accuracy across the pipeline 
  

• Test
  agentic workflows and validate AI-agent decisions, tool usage,
  fallback behavior, exception handling, and human-review
  handoffs 
  

• Define
  and execute validations for AI-generated outputs, including checks for
  missing, incorrect, inconsistent, unsupported, fabricated, or
  policy-violating information 
  

• Validate
  AI outputs against defined business rules, data models, guardrails,
  expected outcome ranges, and acceptance thresholds 
  

• Determine whether
  issues originate in source data, migration logic, data models, integration
  layers, AI-agent behavior, guardrails, or front-end
  presentation 
  

• Develop
  integration and regression tests covering common, negative, edge-case, and
  AI-output validation scenarios 
  

• Work
  directly with US-based Product, Data Intelligence, Engineering, and
  business teams 
  

• Clearly
  communicate defects, evidence, quality risks, guardrail gaps, and test
  findings to stakeholders
  

Requirements

Required Experience and Skills 

• 1–3
  years of QA experience, with a strong focus on API, integration, data
  validation, or migration testing 
  

• Proven
  experience validating data integrity, completeness, accuracy,
  and transformation during migration testing 
  

• Ability
  to understand what data is being used, trace it from source to target,
  and validate it as it moves through system workflows 
  

• Strong
  experience validating complex data models, API payloads, JSON
  structures, mappings, and schema transformations across multiple
  systems 
  

• Experience
  testing AI, LLM, or agentic AI applications, including non-deterministic
  and rules-driven outcomes 
  

• Exposure
  to agentic testing approaches, including validating AI-agent decisions,
  tool usage, fallback behavior, and workflow outcomes 
  

• Experience
  defining or validating guardrails, acceptance criteria, and
  validation checks for AI-generated outputs 
  

• Strong
  API testing experience using tools such as Postman 
  

• Ability
  to create detailed test cases focused on data integrity, mapping,
  transformation, AI-output validation, and edge-case scenarios 
  

• Strong
  functional, integration, regression, analytical, investigative, and
  defect-isolation skills 
  

• Strong
  verbal and written communication skills, with the ability to work directly
  and independently with US-based stakeholders 
  

• Ability
  to explain data-integrity issues, AI behavior, guardrail failures,
  and non-deterministic outcomes to technical and non-technical
  stakeholders 
  

Preferred Experience 

• Insurance
  domain experience, preferably in commercial insurance, quoting,
  underwriting, or insurance application workflows 
  

• Familiarity
  with Model Context Protocol (MCP) 
  

• Experience validating structured
  data consumed or generated by AI systems 
  

• Understanding
  of AI evaluation methods, acceptable outcome ranges, hallucination checks,
  validation thresholds, and guardrail effectiveness 
  

• SQL
  or similar data-querying skills