Technical Lead - Data Processing Infrastructure / Data Platform

We are looking for a Technical Lead to define and build a scalable, user-friendly compute platform for Simulation and ML teams (Perception, Prediction, Planner). You will set the strategic vision for data processing infrastructure, ensuring its scalability, robustness, and alignment with team requirements. You'll make key decisions, prototype solutions, write code, improve frameworks, and provide clear documentation and examples to support adoption.

• Closely collaborate with internal teams (Simulation and ML, including Perception, Prediction, and Planner) to deeply understand their requirements and use cases.


• Evaluate, integrate, and enhance open-source (Apache Spark, Ray, Apache Beam, Argo) and internal tools.


• Prototype, develop, and refine scalable compute solutions by writing code and validating them in practical scenarios.


• Lead and coordinate infrastructure engineering teams.


• Define and communicate the technical vision clearly, providing mentorship and creating documentation to support platform adoption.

• Strong proficiency in Python (required); C++ experience is highly desirable.


• Hands-on experience with distributed computing frameworks (Apache Spark, Ray, Apache Beam, Apache Airflow, Argo Workflows, Kubernetes).


• Proven leadership in architecture design and development of data processing infrastructures.


• Excellent communication and collaboration skills.

• Understanding of ML/AI and experience building ML pipelines.


• Experience with sensor data (camera, lidar, radar) from autonomous vehicles or robotics.


• Experience optimizing costs and performance of distributed computing systems.