Databricks Data Engineer

Databricks Data Engineer / Tech Lead – Azure (Contract)

London (Hybrid) 3 days in the office | 6 months

£620-820 per day (Inside IR35) Umbrella Company Rate

London based trading firm are seeking 2 x experienced Azure Databricks specialist to join a Data team delivering a proof‑of‑concept focused on adopting Databricks and assessing its value across the business.

You will play a key role in designing and implementing Databricks solutions on Azure, building a modern Lakehouse architecture, and integrating on‑premises data with cloud platforms. The role involves working with batch and streaming data, demonstrating near real‑time processing, and embedding governance, lineage, and data quality best practices.

Key Responsibilities

• • Lead Databricks PoC implementation and evaluate platform capabilities
  • Design and build Lakehouse architecture using Delta Lake
  • Integrate on‑prem and Azure data sources
  • Develop batch and streaming pipelines using Spark and Structured Streaming
  • Implement data governance, lineage, and quality controls
  • Collaborate with analytics, data science, and governance teams in an agile environment

Required Experience

• • Strong hands‑on experience with Azure Databricks and Apache Spark
  • Lakehouse architecture and Delta Lake expertise
  • Streaming data (Structured Streaming, Kafka/Event Hubs)
  • Strong SQL, data modelling, and pipeline design skills
  • Azure ADLS Gen2, security, and CI/CD experience

Excellent contract opportunity with high visibility and impact.