Product Manager, Model Gateway

Location: San Francisco, CA
Company Stage of Funding: Series C AI Infrastructure Company ($10B Valuation)
Office Type: Onsite (5 Days Per Week)
Salary: $180,000–$300,000 + Equity

Company Description

As a Product Manager for the Model Gateway platform, you'll own one of the company's most critical internal products: the shared infrastructure layer that routes every AI request across the organization. You'll define how engineering teams access frontier models, optimize reliability and cost, and establish the policies that allow dozens of product teams to build quickly while efficiently sharing AI infrastructure. This is a highly technical platform product role with broad organizational impact.

What You Will Do

• Own the strategy, roadmap, and execution for the company's shared LLM gateway platform.


• Define product requirements for model routing, priority scheduling, rate limiting, usage policies, and access controls.


• Design scalable systems that balance latency, throughput, reliability, and infrastructure costs across multiple AI workloads.


• Build transparent cost attribution and usage reporting that enables teams to understand and optimize model consumption.


• Partner closely with Infrastructure Engineering to translate platform constraints into scalable product capabilities.


• Develop policies that govern shared AI resources while enabling product teams to move quickly.


• Analyze usage patterns, infrastructure metrics, and business outcomes to prioritize platform investments.


• Align engineering, finance, infrastructure, and product stakeholders around shared capacity planning and platform governance.


• Build dashboards, write queries, and prototype lightweight solutions to validate ideas and accelerate execution.


• Continuously evolve the platform as new AI models, workloads, and organizational needs emerge.

Ideal Background

• 5+ years of Product Management experience, preferably building platform, infrastructure, developer, or internal-facing products.


• Previous experience as a Software Engineer, Machine Learning Engineer, Data Scientist, or other highly technical role strongly preferred.


• Strong understanding of distributed systems, platform architecture, APIs, and cloud infrastructure.


• Experience working with AI infrastructure, LLM platforms, or large-scale backend systems.


• Ability to reason about system tradeoffs involving latency, throughput, rate limits, queues, quotas, and reliability.


• Strong analytical skills with experience building dashboards, writing SQL, and using data to drive product decisions.


• Proven ability to lead highly cross-functional initiatives spanning multiple engineering organizations.


• Excellent written and verbal communication skills with experience aligning technical and business stakeholders.


• High ownership mindset with the ability to independently drive ambiguous platform initiatives.

Preferred

• Experience building internal developer platforms or infrastructure products.


• Familiarity with LLM gateways, model routing, AI inference infrastructure, or distributed compute platforms.


• Experience managing infrastructure costs, unit economics, and resource allocation at scale.


• Strong understanding of capacity planning, workload prioritization, and platform governance.


• Experience balancing technical constraints with business objectives in rapidly growing organizations.


• Startup experience building foundational platform capabilities from the ground up.


• Comfortable prototyping lightweight technical solutions using AI-assisted development tools.


• Passion for AI infrastructure, platform engineering, and enabling high-performing engineering organizations.

Compensation and Benefits

• Base salary: $180,000–$300,000.


• Generous equity package vested over four years.


• Bi-annual performance bonus.


• Up to $15,000 relocation assistance.


• $10,000 housing bonus for employees living within 0.5 miles of the office.


• $1,500 monthly meal stipend.


• Complimentary Equinox membership.


• $200 monthly laundry reimbursement.


• $200 monthly wellness reimbursement.


• Comprehensive medical, dental, and vision insurance.


• Opportunity to own the AI infrastructure platform that powers every product across one of the world's fastest-growing AI companies, directly influencing engineering velocity, infrastructure efficiency, and the future of enterprise AI development.