Lead Data Engineer

Talworx is an emerging recruitment consulting and services firm , we are hiring for our Fintech Product based client, In this role, you will be working with a team or teams of enthusiastic members supporting our critical technology systems and guiding our business partners & end users with industry best practices, solution design, and creating long term value to our customers.

engaging work environment. You will work in a (truly) global team and encouraged for thoughtful risk-taking and self-initiative.

Job Summary:

As a member of the Cognitive Engineering team, you will build and maintain enterprise-scale data extraction, automation, and ML model deployment pipelines. You will design resilient, production-ready systems within an AWS-based ecosystem, collaborating with a hands-on, technically strong global team to solve high-complexity problems end-to-end.

Key Responsibilities:

• Develop, deploy, and operate data extraction and automation pipelines in production.
• Integrate and deploy machine learning models into pipelines (e.g., inference services, batch scoring).
• Lead the delivery of complex extraction, transformation, and ML deployment projects.
• Scale pipelines on AWS (EKS, ECS, Lambda) and manage DataOps processes with Celery, Redis, and Airflow.
• Implement robust CI/CD pipelines on Azure DevOps and maintain comprehensive test coverage.
• Strengthen data quality and reliability through logging, metrics, and automated alerts.
• Partner with data scientists, ML engineers, and product teams to align on requirements and delivery timelines.

Technical Requirements:

• 2-6 years of relevant experience in data engineering, automation, or ML deployment.
• Expert proficiency in Python, including building extraction libraries and RESTful APIs.
• Hands-on experience with task queues and orchestration: Celery, Redis, Airflow.
• Strong AWS expertise: EKS/ECS, Lambda, S3, RDS/DynamoDB.
• Containerization and orchestration experience: Docker (mandatory), basic Kubernetes (preferred).
• Proven experience deploying ML models to production.
• Solid understanding of CI/CD practices and hands-on experience with Azure DevOps.
• Familiarity with SQL and NoSQL stores (e.g., PostgreSQL, MongoDB)