MLOps / Serving Engineer

• Deploy
  fine-tuned LLMs using vLLM, TensorRT-LLM, or Triton with continuous
  batching on AWS GPU instances
  
• Build
  shadow-mode deployment: run fine-tuned model alongside production, log
  comparison data without impacting live traffic
  
• Execute
  staged rollout: canary (5%) → gradual ramp
  (25% → 50% → 100%) with automated rollback on quality
  degradation
  
• Optimize
  inference for input-heavy workloads (~17K token inputs, ~130 token
  outputs): prefill throughput, KV-cache, INT8 quantization
  
• Build
  monitoring dashboards: latency, throughput, accuracy metrics, cost per
  request
  
• Design
  auto-scaling; implement high-availability (2× instances); automated
  rollback triggers on end-to-end quality metrics
  

Requirements

Benefits