Sr Software Engineer - Quality

• Drive the technical direction and feature roadmap of large-scale cloud systems
• Serve as a technical lead contributing to and directing the efforts of development teams, including internal and external team members
• Determines operational feasibility by evaluating analysis, problem definition, requirements, solution development, and proposed solutions
• Drives detailed definition of feature requirements through discussions with product managers and customers
• Supports continuous improvement by investigating alternatives and new technologies
• Prepares and installs solutions by determining and designing system specifications, standards, and programming
• Evaluates code to ensure that it is valid, is properly structured, meets industry standards, and is compatible with browsers, devices, or operating systems
• Mentor Software Engineers to allow for skill/knowledge development through advice, coaching, and training opportunities
• Update job knowledge by studying state-of-the-art development tools, programming techniques and computing equipment; participating in educational opportunities; reading professional publications; maintaining personal networks; participating in professional organizations
• May perform other job duties as directed by Employee's Leaders

• Advanced knowledge of software development methodologies, practices, concepts, and technologies obtained through formal training and / or work experience
• Proactively shares knowledge and skills of software development and testing to solve a range of problems
• Advanced knowledge of at least one required programming language
• Excellent partnering, communication, negotiation skills, and execution in working with various Technology or partner Teams
• Ability to articulate (verbally / visually) ideas, approach, concepts, options, risk, impact, and provides guidance to technical, non-technical, and Leadership audiences
• Ability to lead multiple assignments, whether administrative or project related, while maintaining a successful level of completion in all responsible work
• Ability to mentor others
• Ability to lead ideating, presentations, mentoring and group-thinking sessions
• Ability to influence without authority

Education

• Required: Bachelor's degree in Computer Science, Engineering, Information Systems or related field and/or equivalent formal training

Experience

• Required: Advanced-level experience, seasoned and specialized knowledge in:

  • Software development and methodologies

  • 5-7 years of relevant work-related experience

  • 5+ years in Software QA/Testing, with 2+ years in AI/ML or LLM-based systems; hands-on experience testing agentic/multi-agent architectures

  • Strong programming skills in Python or TypeScript/JavaScript; experience building test harnesses, simulators, and fixtures.

  • Experience with LLM evaluation (exact/soft match, BLEU/ROUGE, BERTScore, semantic similarity via embeddings), guardrails, and prompt testing.

  • Agentic AI performance/ Load testing experience and Agentic AI defect life cycle and defect triage experience

  • Expertise in distributed systems testing, latency profiling, resiliency patterns (circuit breakers, retries), chaos engineering, and message queues.

Preferred: Experience in

• Prior ownership of cost/latency/SLAs for AI workloads in production
• Knowledge of MLOps (model versioning, datasets, evaluation pipelines) and A/B experimentation for LLMs.
• Background in cloud (AWS), serverless, containerization, and event-driven architectures.
• Multi-agent simulators, agent graph testing, and tooling latency emulation
• Building test harnesses, simulators, and fixtures

Other Qualifications

• Must meet confidentiality expectations as to confidential, proprietary and sensitive Company information

• Ability to work extended hours as needed