Data Engineer

We're looking for a talented Data Engineer to play a key role in designing, building, and optimizing data pipelines, enabling seamless data integration and transformation. You will work extensively with Azure Data Factory (approximately 70% of the time), ensuring the efficient movement and processing of data across multiple systems. In this role, you'll collaborate closely with data architects, engineers, and business stakeholders to drive data-driven decision-making in a fast-paced environment.

You will:

• Design, develop, and maintain scalable data pipelines using Azure Data Factory (ADF).
• Integrate data from multiple sources, both structured and unstructured, ensuring efficient data flow.
• Optimize data transformation processes to enhance performance and reduce latency.
• Implement monitoring, logging, and automated testing for data pipelines.
• Work with stakeholders to understand business requirements and translate them into technical solutions.
• Ensure data quality, consistency, and compliance with industry standards and best practices.
• Troubleshoot and resolve data pipeline issues, ensuring high availability and reliability.
• Mentor junior data engineers by sharing best practices and conducting code reviews.

About You:

• Ideally 5+ years of experience in data engineering with a strong focus on building ETL/ELT pipelines
• Extensive hands-on experience with Azure Data Factory (ADF) for data orchestration and transformation.
• Proficiency in SQL and NoSQL databases for data modelling and querying.
• Experience with other Azure data services such as Azure Synapse, Azure Data Lake, and Azure Databricks.
• Strong programming skills in Python, Scala, or another data-focused language.
• Familiarity with data governance, security, and compliance best practices
• Experience working with messaging systems such as Azure Service Bus or Kafka.
• Strong problem-solving skills and ability to work independently and collaboratively.
• Excellent communication skills, with the ability to convey complex technical concepts to non-technical stakeholders.