QA Engineer (Automation & AI Quality) (100% Remote) (Pakistan Only)

About AllShore Talent

AllShore Talent is a leading remote staffing company, offering top-tier professionals working 100% remote to businesses worldwide. Specializing in IT and software development, design, administrative support, digital marketing, and more. AllShore connects organizations with skilled talent to meet diverse business needs.

About The Client
The Client is a full-service digital solutions partner committed to building websites and apps that are as unique as the businesses they represent. With a deep belief in the value of strong client relationships, their approach is grounded in trust, collaboration, and a genuine understanding of each client’s goals. From custom web and app design to comprehensive eCommerce strategies, branding, marketing, and social media management, they offer tailored solutions that help businesses grow and thrive online. Whether supporting startups, medical practices, or real estate firms, they reject cookie-cutter templates in favor of scalable, intuitive, and beautifully designed platforms. Their team works closely with each client, providing hands-on support and flexible service options that make managing an online presence easy, impactful, and aligned with business objectives.

About The Role
We are looking for a QA Engineer to establish and execute a comprehensive testing strategy for our AI-powered platform. You will ensure high-quality releases by developing automated and manual testing processes across backend, frontend, and AI-driven components.

Project context
You will establish a comprehensive test strategy across API, UI, and AI workflows for a multi tenant, white label RAG platform.

What you will do

• Define the test plan across unit, integration, end to end, and performance levels

• Build and maintain automated tests for APIs and UI, Pytest, Playwright or Cypress

• Create test data and fixtures for RAG pipelines, embeddings, and vector indexes

• Add evaluation and regression checks for LLM behavior, prompt and retrieval quality

• Implement observability in tests, logs, traces, and metrics linked to releases

• Partner with engineers to add quality gates in CI CD and track defects to closure

What you will bring

• 3 plus years in QA engineering with automation

• Strong API testing, Postman or equivalent, and UI automation experience

• Familiarity with Python based backends and SQL databases

• Comfort testing AI features, or a strong interest and ability to learn quickly

Nice to have

• Experience with load testing, k6 or Locust

• Exposure to LLM evaluation tools, LangSmith, RAGAS, or custom harnesses

• Security and privacy testing basics