Video Games Reviewer (AI Evaluation)

Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.

Role Overview

We are looking for an experienced Video Games Domain Reviewer to support the quality assurance of data used to evaluate Large Language Models. The role involves reviewing gaming-related prompts and completed tasks, checking factual accuracy, reasoning quality, completeness, and guideline compliance, and providing clear, evidence-based feedback.

The work may cover game development, game design, esports, gaming platforms, publishers, genres, gaming communities, and current industry trends.

CONTRACT: Contractor assignment, 8 weeks

COMMITMENT: Full-time, 40 hours per week with at least 4 hours of PST overlap

LOCATIONS: Remote - USA

Key Responsibilities

• Review and validate video game-focused prompts across a broad range of gaming topics.
• Evaluate completed tasks for factual accuracy, reasoning quality, completeness, and relevance.
• Identify factual errors, hallucinations, logical inconsistencies, and weak annotations.
• Verify claims related to games, publishers, platforms, genres, esports, and game development.
• Assess whether prompts are sufficiently challenging and aligned with project objectives.
• Ensure compliance with project instructions, review rubrics, and quality standards.
• Provide clear, constructive, and evidence-based feedback to contributors.
• Document review findings and escalate ambiguous or complex cases when required.
• Collaborate with project managers and AI teams to improve evaluation quality and workflows.

• Master’s degree or higher in Game Design, Interactive Media, Computer Science, or a closely related discipline.
• At least 3 years of professional, research, teaching, or industry experience in the video games domain.
• Strong knowledge of game development, gaming platforms, publishers, genres, esports, or gaming communities.
• Experience reviewing, evaluating, editing, or quality-checking technical or domain-specific content.
• Ability to identify subtle factual, logical, and contextual errors.
• Excellent written English communication skills.
• Strong analytical ability and attention to detail.
• Ability to work independently in a quality-focused remote environment.