Senior Data Engineer Real-Time ML Infrastructure- 1469

Must Have:

• Spark or Flink or Beam or Kafka Streams
  
• Real-time streaming pipeline experience
  
• SQL and large-scale schema design
  
• AWS or equivalent cloud platforms
  
• Python, Java, or Scala
  
• ML workflow support (feature engineering, data validation, etc.)

Responsibilities:

• Design, build, and maintain scalable batch and real-time data pipelines for user interaction data, metadata, and model features
  
• Develop and operate offline and real-time feature stores to support ML inference and training workflows
  
• Partner with ML engineers to define data schemas, validation logic, and build production-ready datasets
  
• Implement monitoring and observability for data quality and pipeline reliability
  
• Optimize data workflows for speed, cost, and scalability across large-scale datasets
  
• Translate personalization requirements into robust data infrastructure solutions
  
• Participate in selection and adoption of modern data tooling and cloud-native technologies
  
• Contribute to team’s technical excellence through code reviews and design discussions

Qualifications:

• Bachelor’s or Master’s in Computer Science, Data Engineering, or related technical field
  
• 5+ years of experience building production-grade distributed data systems
  
• Expertise in modern data frameworks (Spark, Flink, Beam, Kafka Streams)
  
• Proficient in SQL and schema design for high-scale data environments
  
• Strong programming skills in Python, Java, or Scala
  
• Experience with cloud platforms such as AWS and infrastructure components like data lakes, warehouses, and feature stores
  
• Proven ability to support machine learning data workflows and pipelines
  

Preferred:

• Experience in building ML infrastructure for personalization or recommendation systems
  
• Familiarity with MLOps tools (MLflow, TFX, Kubeflow)
  
• Hands-on knowledge of real-time ML serving and online feature generation
  
• Prior work in early-stage or 0→1 product development environments
  
• Strong cross-functional collaboration with ML and product teams