Senior Data Engineer

Job Description

We are seeking an experienced Senior Data Engineer to design, develop, and maintain scalable cloud-native data platforms that support enterprise analytics and AI initiatives. You will be responsible for architecting modern data solutions, building robust data pipelines, and driving cloud transformation while ensuring data quality, security, and operational excellence.

This role offers the opportunity to work with modern Data Lakehouse architectures, cloud-native technologies, and enterprise AI platforms to enable data-driven decision making across the organisation.

Key Responsibilities

Data Platform Architecture

• Design, develop, and implement scalable, secure, and highly available cloud-native data architectures.
• Lead the implementation and enhancement of enterprise Data Lakehouse solutions.
• Collaborate with business stakeholders, solution architects, and development teams to translate business requirements into scalable technical solutions.
• Produce and maintain technical architecture documentation, solution designs, and implementation standards.

Data Pipeline Development

• Design, build, and maintain scalable ETL/ELT pipelines using modern data engineering frameworks and cloud technologies.
• Optimise data ingestion, transformation, and processing workflows for performance, scalability, reliability, and cost efficiency.
• Implement automated data quality validation, monitoring, and alerting to ensure data integrity and reliability.
• Support continuous improvement of data engineering processes and platform capabilities.

Data & AI Solution Architecture

• Design and architect enterprise Data & AI solutions that align with business objectives and cloud-first strategies.
• Lead cloud migration initiatives and support deployment across multi-cloud environments where required.
• Integrate Data Engineering, Analytics, and AI capabilities into enterprise data platforms.
• Evaluate emerging technologies and recommend modern cloud-native solutions to improve platform capabilities.

Cloud Platform Engineering

• Build and manage cloud-native data infrastructure using cloud platform services.
• Implement Infrastructure as Code (IaC) practices using Terraform or equivalent technologies.
• Ensure compliance with enterprise security standards, data governance policies, and regulatory requirements.
• Support platform operations, monitoring, troubleshooting, and performance optimisation.

Technical Leadership

• Provide technical leadership and mentorship to junior data engineers.
• Participate in architecture reviews and contribute to enterprise data engineering standards and best practices.
• Collaborate with cross-functional teams including software engineers, data scientists, DevOps engineers, and business stakeholders.

Requirements

• Bachelor's Degree in Computer Science, Information Technology, Computer Engineering, Data Engineering, or a related discipline.
• Minimum 3 years of hands-on experience in Data Engineering, Data Platform Architecture, or Data Integration at enterprise or production scale.
• Strong experience designing and implementing scalable cloud-based data platforms.
• Solid understanding of cloud computing principles, including Infrastructure as Code (IaC), containerisation, microservices architecture, cloud networking, security, identity and access management (IAM), and distributed systems.
• Strong proficiency in SQL, Python, and Apache Spark.
• Hands-on experience developing ETL/ELT pipelines and large-scale data processing solutions.
• Experience with Apache Kafka, Apache Airflow, or similar workflow orchestration and streaming technologies.
• Knowledge of DataOps, Data Lakehouse architectures, metadata management, and data governance best practices.
• Familiarity with AI/ML platforms, MLOps, or LLMOps concepts.
• Experience translating business requirements into scalable technical solutions.
• Excellent analytical, problem-solving, communication, and stakeholder management skills.

Preferred Qualifications

• Hands-on experience with Amazon Web Services (AWS) cloud services.
• Professional cloud certifications such as AWS Certified Solutions Architect – Professional, AWS Certified Data Engineer – Associate, or equivalent.
• Experience with cloud-native Data & AI services, including: Amazon SageMakerAmazon QuickSightAmazon S3AWS GlueAWS Lake FormationAWS BedrockAmazon AgentCore
• Experience implementing machine learning pipelines and MLOps practices.
• Familiarity with serverless computing, edge computing, and IoT architectures.
• Experience with data governance frameworks, metadata management, and data cataloguing solutions.
• Knowledge of Business Intelligence (BI) and data visualisation platforms such as Power BI, Tableau, or Amazon QuickSight.
• Experience working within Agile, DevOps, or CI/CD delivery environments.