German Language Evaluator - Expert

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.

Position: Bilingual German Generalist Evaluator Expert
Type: Contract
Compensation: $25–$30/hour
Location: Remote
Duration: 2–4 months
Commitment: 20+ hours/week

Role Responsibilities

• Design detailed prompts in German and/or English with constraints and instructions for real-world relevance in Austria, Switzerland, and Germany.
• Define evaluation standards and develop rubrics considering linguistic nuance and cultural conventions in these regions.
• Test models by running prompts and assess outputs for accuracy, fluency, and cultural fit in German and English.
• Collaborate in quality assurance to ensure consistency and reliability across German-language benchmarks.
• Work independently and meet deadlines while contributing to a structured project environment.

Qualifications

Must-Have

• Native-level fluency in German specific to Austria, Switzerland, or Germany.
• Must be native to Austria, Switzerland, or Germany with deep cultural familiarity.
• BS or BA from a reputable institution.
• Strong writing and critical thinking skills.
• Familiarity with ChatGPT or similar tools.
• Based in Austria, Switzerland, or Germany.

Preferred

• Experience in teaching, research, editing, or academic writing.
• Experience creating evaluation criteria or rubrics.
• Familiarity with LLMs, prompting, or model evaluation.

Application Process (Takes 20–30 mins to complete)

• Complete an AI-led interview (about 15 minutes).
• If approved, complete a paid assessment focused on writing and rubric creation.
• If selected, you will be invited to work on the project.

Resources & Support

• For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome
• For any help or support, reach out to: [email protected]

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.