Senior Data Engineer

£45,000 to 75,000 GBP

Hybrid WORKING

Location: Central London, Greater London - United Kingdom Type: Permanent

Senior Data Engineer

Location: London - Hybrid (3 days per week in the office)

Salary: £45,000-£70,000 DOE, Various levels available

Security requirement: Sole UK National / UK Citizen only, due to the level of security clearance required.

The Opportunity

We are looking for an experienced Senior Data Engineer with strong AWS expertise to join a specialist technology team delivering complex data engineering solutions across national security, government and public-sector environments.

You will design, build and maintain scalable data pipelines and cloud-based data platforms, working alongside data architects, data scientists, analysts and business stakeholders to turn complex data into reliable, accessible and actionable information.

This role combines hands-on engineering with technical leadership. You will help shape data solutions, guide engineering delivery and ensure teams follow best practices across software engineering, DataOps, security and data management.

Key Responsibilities

• Design, build and maintain scalable data pipelines and ETL/ELT workflows.
• Develop and manage cloud-based data infrastructure using Amazon Web Services (AWS).
• Apply software engineering and DataOps best practices to deliver reliable, high-quality and maintainable solutions.
• Work with stakeholders to understand business requirements and translate them into effective technical solutions.
• Collaborate with data architects, scientists, analysts and multidisciplinary engineering teams.
• Provide technical leadership to data engineering teams and establish effective delivery frameworks.
• Lead technical design sessions and contribute to the selection of data platforms, products and tools.
• Define work packages and coordinate technical delivery to ensure successful outcomes.
• Implement appropriate approaches to data quality, security, governance, orchestration and data modelling.
• Use version control, automated testing and CI/CD practices to support efficient deployment.
• Communicate technical concepts clearly to technical and non-technical stakeholders.

Essential Skills and Experience

• Proven experience in data engineering, including building and maintaining production-grade data pipelines.
• Strong hands-on experience with AWS and its data or cloud infrastructure services.
• Proficiency in Python and SQL, or comparable programming languages.
• Experience with ETL/ELT processes, data orchestration and data transformation.
• Understanding of data modelling, data management, data security and data quality.
• Experience with software engineering principles, version control and CI/CD.
• Experience working with business stakeholders to define requirements and deliver technical solutions.
• Ability to provide technical guidance and lead engineering delivery.
• Strong communication, problem-solving and collaboration skills.

Desirable Experience

• Experience with AWS services such as S3, Glue, Lambda, Step Functions, Redshift or related data engineering technologies.
• Familiarity with Apache Spark, Airflow, distributed processing, data lakes and data warehouses.
• Experience with infrastructure as code, Docker or automated deployment pipelines.
• Background in government, defence, national security or other highly regulated environments.

Package and Benefits

• Salary of £45,000-£70,000, depending on experience.

Security Clearance Requirements

Applicants must be sole UK citizens (sole UK nationals) and eligible to obtain the required level of UK security clearance. Candidates must be willing to undergo the relevant clearance process.

If you have strong AWS data engineering experience and are looking to take on technically challenging work with opportunities to develop your leadership skills, we would welcome a conversation.