Data Engineer

• Design, develop, and optimise ETL/ELT pipelines for structured and unstructured data.
  
• Build and manage data lakes, data warehouses, and real-time data streaming solutions.
  
• Integrate data from multiple sources (databases, APIs, files, external feeds) using tools like Kafka, MinIO, and message brokers.
  
• Work with distributed storage and compute platforms (Clickhouse, Apache Pinot, Hadoop, Spark, etc.).
  
• Implement data modelling, data quality, and data governance best practices.
  
• Collaborate with DevOps for CI/CD, monitoring, and automation (Docker, Jenkins, Azure DevOps).
  
• Ensure data security, privacy, and compliance with regulatory standards.
  
• Troubleshoot and resolve data pipeline and infrastructure issues.
  
• Document data flows, architecture, and technical specifications.
  

Requirements

• 5+ years of experience in data engineering or related roles.
  
• Strong programming skills in Python, Java, or Scala.
  
• Experience with distributed data systems: Kafka, Clickhouse, Pinot, MinIO, Hadoop, Spark, or similar.
  
• Proficiency in SQL and working with relational (MS SQL, PostgreSQL, Oracle) and NoSQL databases.
  
• Hands-on experience with ETL/ELT tools and data integration frameworks.
  
• Experience with DevOps tools: Docker, Jenkins, Git, Azure DevOps.
  
• Understanding of data modelling, data warehousing, and real-time data processing.
  
• Good communication and teamwork skills.
  

• Experience in BFSI, capital markets, or fintech domains.
  
• Knowledge of data governance, data quality, and regulatory compliance.
  
• Certifications in relevant data engineering, cloud, or big data technologies.