Databricks/Python Specialist

Requirements

• Experienced Data Engineer with deep expertise in Azure Databricks and Python/PySpark to design, develop, and maintain robust data processing solutions.
  


• Focuses on building reliable and scalable data processing capabilities within a Databricks environment, rather than traditional BI or data enrichment solutions.
  


• Strong hands-on experience managing complex data workloads, designing efficient data models, and ensuring the stability, quality, and maintainability of data solutions within a modern cloud architecture.
  


• Strong hands-on experience with Azure Databricks as the primary technology platform.
  


• Deep expertise in Python, PySpark, and SQL.
  


• Proven experience designing and managing data processing solutions within Databricks environments.
  


• Experience handling both streaming and batch data processing workloads.
  


• Strong understanding of data modeling principles within Databricks.
  


• Experience managing multiple concurrent data processing pipelines operating on shared data sources.
  


• Hands-on experience with Azure cloud services and CI/CD pipeline implementation using YAML.
  


• Experience with Infrastructure as Code (IaC) using ARM templates and/or Bicep.
  


• Experience working within Agile development environments.
  


• Understanding of end-to-end data platform and data solution architectures.
  


• Strong focus on quality, reliability, maintainability, and minimizing technical debt.
  


• Proactive mindset with a commitment to continuous improvement.
  


• Excellent collaboration and communication skills.
  


• Structured and pragmatic engineering approach with a focus on practical, sustainable solutions.
  


• Ability to make sound technical decisions within complex and evolving environments.
  
  

Benefits