Databricks Data Engineer/DWH Developer

Senior Databricks Data Engineer/DWH Developer

Location: Bucharest

Contract Type: B2B

Responsibilities:

Databricks Development & Architecture

• Design and implement robust, scalable, and high-performance ETL/ELT pipelines using PySpark/Scala and Databricks SQL.

• Develop and maintain Lakehouse architecture, combining the best practices of DWH and Data Lake.
• Optimize Databricks clusters, Spark operations, and Delta tables (e.g., Z-ordering, Compaction, Query Tuning) to reduce latency and compute costs.
• Design and implement real-time and near-real-time streaming solutions using Spark Structured Streaming and Delta Live Tables (DLT).
• Develop and manage complex workflows using Databricks Workflows or external orchestration tools (Azure Data Factory, Airflow).

• Integrate Databricks pipelines into CI/CD processes using Git, Databricks Repos, and Bundles.

Technical Skills & Qualifications:

• Extensive experience with the Databricks platform (Workspace, Cluster Management, Notebooks, Databricks SQL).
• Deep understanding of Apache Spark architecture (RDD, DataFrames, Spark SQL) and advanced optimization techniques.
• Advanced expertise in Delta Lake (ACID, Time Travel, Merge, Optimize, Vacuum).

• Advanced programming in Python (PySpark) and/or Scala (Spark).

• Strong SQL and data modeling skills (Dimensional, 3NF, Data Vault).
• Solid experience with a major Cloud platform (AWS, Azure, or GCP), including storage services (S3, ADLS Gen2, GCS) and networking.