What You'll Do
Design data slices and explore data shapes that expose meaningful model failure modes across domains, including finance, code, and enterprise workflows
Build and refine evaluation rubrics and reward signals for RLHF and RLVR training pipelines
Model annotator behavior and run experiments to improve different model capabilities
Develop quantitative frameworks for measuring dataset quality, diversity, and downstream impact on model alignment and capability
Partner with lab research teams to translate their training objectives into concrete data and evaluation specifications
Move fast from hypothesis to experiment, extract actionable insights from messy results, and iterate quickly
REQUIREMENTS
Must-Have
Strong quantitative instincts with familiarity with LLM training pipelines, RLHF or RLVR, or evaluation methodology. Does not need a PhD but must have the research depth of a strong undergrad or master's researcher
Genuine obsession with how data structure, selection, and quality drive model behavior. This is the core of the work and must be intrinsically motivated
Ability to design lightweight experiments, move fast, and extract actionable insights from messy and incomplete results
Comfort working across domains, the work touches finance, software engineering, policy, and more. Must be able to context-switch and reason clearly across all of them
Bias toward building over theorizing. Ships experiments and iterates, does not get stuck in design
Nice-to-Have
Prior work or internship at RL environment companies, AI safety organizations, or benchmarking organizations such as METR or Artificial Analysis
Background in evaluation methodology, benchmark design, or dataset curation at a lab or research organization
Exposure to annotator modeling, reward signal design, or alignment-related research
The standard base is 150 to 250k, but they also engage in profit sharing, so their total cash comp will land between 250 and 450k, and then, of course, there's equity on top of that.
Salary :
Your email job alert is now active!