Tetriz is an AI Engineering Intelligence platform that helps engineering organizations become AI-native, faster. We measure how AI coding tools like Cursor, Claude Code, and GitHub Copilot are actually used, improve engineer effectiveness through prompt and workflow coaching, and help engineering leaders demonstrate AI ROI with board-ready, defensible insights.
A builder mindset is the core of this role and where you'll spend most of your time. But we're a small team building a whole product, not a research lab. The best person here treats ML systems as their primary craft while staying willing to do whatever the product needs — thinking through the product itself, shipping backend or frontend code, untangling data pipelines. We're looking for someone energized by the breadth, not someone who wants to stay in their lane.
What you'll work onEvaluation systems for AI featuresHelp build the eval backbone our AI features ship against — failure taxonomies, LLM-as-judge rubrics, golden datasets, calibration against human judgment.Learn what it takes to keep automated scores honest as models and prompts change. A feature with no eval has no quality floor.
Model routing & inference economicsGet hands-on with how we route work across models — balancing cost, quality, and latency per task.Help run the experiments that justify those choices and catch regressions.
Scoring, measurement & signal qualityWork on turning noisy, real-world signals into scores you can actually trust — grounded in real statistical rigor, not vibes.Help move heuristic-driven approaches toward calibrated, monitored systems.
MLOps & productionGet exposure to the full lifecycle — feature pipelines, model versioning, rollout, monitoring for drift and silent quality decay.Work alongside engineering to see how models get served reliably at low latency.
What we're looking forMust have
A builder mindset is the core of this role and where you'll spend most of your time. But we're a small team building a whole product, not a research lab. The best person here treats ML systems as their primary craft while staying willing to do whatever the product needs — thinking through the product itself, shipping backend or frontend code, untangling data pipelines. We're looking for someone energized by the breadth, not someone who wants to stay in their lane.
What you'll work onEvaluation systems for AI featuresHelp build the eval backbone our AI features ship against — failure taxonomies, LLM-as-judge rubrics, golden datasets, calibration against human judgment.Learn what it takes to keep automated scores honest as models and prompts change. A feature with no eval has no quality floor.
Model routing & inference economicsGet hands-on with how we route work across models — balancing cost, quality, and latency per task.Help run the experiments that justify those choices and catch regressions.
Scoring, measurement & signal qualityWork on turning noisy, real-world signals into scores you can actually trust — grounded in real statistical rigor, not vibes.Help move heuristic-driven approaches toward calibrated, monitored systems.
MLOps & productionGet exposure to the full lifecycle — feature pipelines, model versioning, rollout, monitoring for drift and silent quality decay.Work alongside engineering to see how models get served reliably at low latency.
What we're looking forMust have
- 1.5–2 years of hands-on experience in Data Science, Machine Learning, Software Engineering, or a related role.
- Experience building and shipping DS/ML systems through professional work, personal projects, research, or open-source contributions — where you've built and run something end to end, not just notebooks.
- Comfort with Python and working SQL knowledge.
- Basic grounding in applied statistics — you can explain what a metric means and when it might be misleading.
- A builder's instinct — genuinely curious about product decisions, backend, or frontend, not just the modeling layer.
- Some exposure to LLMs — prompting, using APIs, or experimenting with model behavior.
Nice to have
- Any exposure to evaluation or observability tooling for LLM features.
- Experience with information retrieval, entity-matching, or record-linkage.
- Interest in developer-productivity, code analytics, or DevEx data.
We aspire to create an inclusive culture of diverse people not just because it's the right thing to do but because heterogeneity inspires us and is more fun! We employ people solely on merit and do not discriminate against any employee or applicant because of race, creed, color, religion, gender, sexual orientation, gender identity/expression
We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses and identifying potential inconsistencies or verification signals in application materials based on available information. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.
Your email job alert is now active!
