Senior Data Engineer

Senior Data Engineer Role (Python / Spark / Databricks) - Hybrid - Contract - Banking

Role - Senior Data Engineer

Rate - £850 p/d (Inside IR35)

Duration - 6 months with very likely extension

Location - Hybrid / Liverpool Street (London) - 3 days per week in a Liverpool Street office

About the Role

Global financial institution seeking a highly skilled Senior Data Engineer to join a growing data engineering function responsible for delivering critical data platforms and analytics capabilities across the business.

This is a genuinely hands-on engineering role, requiring strong expertise in Python, Databricks, Spark/PySpark and SQL. The successful candidate will be actively involved in the design, development, optimisation and support of enterprise-scale data platforms, working alongside engineers, architects and business stakeholders to deliver robust and scalable solutions.

The role offers the opportunity to work on complex data challenges within a modern cloud-based environment, leveraging the latest data engineering technologies and best practices.

Key Responsibilities

Data Engineering & Development

Design, develop and maintain scalable data pipelines using Python, Databricks and PySpark
Build and optimise ETL/ELT solutions processing large-scale datasets
Develop high-quality, reusable code and engineering frameworks
Deliver robust, performant and maintainable data solutions
Troubleshoot and resolve complex production and performance issues
Support the ongoing enhancement and modernisation of enterprise data platformsDatabricks & Data Platform Engineering

Develop and support Databricks-based data solutions and Lakehouse architectures
Utilise Delta Lake and Spark technologies to deliver scalable data processing capabilities
Optimise distributed data processing workloads for performance and reliability
Implement data quality controls and monitoring solutions
Contribute to platform stability, scalability and operational excellenceSolution Delivery

Work closely with business and technology stakeholders to understand requirements and deliver effective solutions
Participate in Agile delivery processes including sprint planning, refinement and estimation
Support the delivery of strategic data initiatives across multiple business areas
Collaborate with architects, engineers and product teams to deliver high-quality outcomes
Identify and resolve technical risks and implementation challengesEngineering Best Practice

Promote software engineering best practices including testing, code reviews and version control
Support CI/CD adoption and deployment automation
Contribute to coding standards and engineering governance
Drive continuous improvement across development and support processes
Mentor junior engineers and contribute to knowledge sharing within the teamRequired Skills & Experience

Strong commercial experience as a Data Engineer or Senior Data Engineer
Expert-level Python development skills
Extensive hands-on experience with Databricks in enterprise environments
Strong Apache Spark and PySpark expertise
Advanced SQL development and optimisation skills
Experience designing and building large-scale ETL and data integration solutions
Strong understanding of distributed computing concepts and data processing frameworks
Experience working with modern cloud platforms including Azure, AWS or GCP
Knowledge of software engineering principles, testing and code quality practices
Experience working within Agile development environments
Excellent problem-solving and troubleshooting capabilitiesDesirable Experience

Experience within investment banking, financial services or other highly regulated environments
Exposure to risk, trading, treasury or regulatory data domains
Experience with Kafka, streaming technologies or real-time data processing
Databricks certifications
Experience with Airflow, Databricks Workflows or similar orchestration tools
CI/CD pipeline development and DevOps practices
Docker and Kubernetes
Experience supporting data science or machine learning workloadsThis role will suit a Senior Data Engineer who enjoys remaining close to the technology and has strong recent experience building production-grade solutions using Python, Databricks, Spark/PySpark and SQL in enterprise environments.

GCS is acting as an Employment Business in relation to this vacancy
Only candidates based in UK and eligible to work in UK are allowed