Senior/Lead Data Engineer

Requirements

Key Responsibilities

• Design, build and optimise enterprise data pipelines and data platforms.
  
• Develop scalable cloud-native data engineering solutions.
  
• Collaborate with business and technical stakeholders to deliver data-driven solutions.
  
• Define and maintain data architecture and engineering standards.
  
• Drive DevSecOps and CI/CD best practices.
  
• Mentor and coach Data Engineers.
  
• Ensure data quality, governance and platform reliability.
  
• Participate in Agile ceremonies and technical planning.
  
• Drive continuous improvement and innovation across the data environment.
  

Requirements

• Bachelor's Degree or Diploma in Computer Science, Engineering, Mathematics, Information Systems or similar.
  
• 8+ years' experience in Data Engineering or Software Engineering.
  
• Strong experience with Business Intelligence and Data Warehousing.
  
• Advanced SQL skills.
  
• Expert knowledge of Python and PySpark.
  
• Experience with Hadoop and Apache Spark.
  
• Strong AWS experience, including EMR, S3, Athena, RDS, Lambda and API Gateway.
  
• Good understanding of SDLC, Solution Architecture and DevSecOps.
  
• Experience with Azure, Google Cloud Platform and Snowflake is advantageous.
  
• Previous experience leading or mentoring engineering teams.
  
• Retail or FMCG industry experience is essential.
  

Technical Skills

• Python
  
• PySpark
  
• SQL
  
• AWS
  
• Hadoop
  
• Apache Spark
  
• Data Warehousing
  
• Business Intelligence
  
• ETL / ELT
  
• Snowflake
  
• Azure
  
• Google Cloud Platform
  
• DevSecOps
  
• CI/CD
  
• Solution Architecture