Databricks Engineer

Proven, hands-on experience with Azure Data Factory and end-to-end pipeline development. Strong experience with Azure Databricks, building notebooks/jobs, and optimizing workloads. Advanced Python skills for data processing, unit testing, and scripting automation. Strong SQL skills: designing and writing mid-to-complex queries, stored procedures, functions, and optimizing query performance (T-SQL).Experience with SQL Server schema design, and working knowledge of SSIS packages (developing, debugging, maintaining). Familiarity with Azure DevOps for source control (Git), CI/CD pipelines, build/release automation, and infrastructure-as-code patterns. Enforce data-quality checks, lineage, and monitoring (alerts, metrics, SLAs) across pipelines. Experience implementing data quality frameworks, monitoring, and alerting. Strong problem-solving skills and a track record of delivering production systems with SLAs. Document architecture, data contracts, and operational procedures; conduct knowledge transfer to other engineers on the team. Education: Bachelor's or Master's degree in Computer Science, Information Technology, or a related field. Experience: 5+ years of experience in data engineering or a similar role. Azure expertise: Proven experience with Azure data services including Azure Data Factory, Azure Synapse Analytics, and Azure Data Lake Storage. Databricks expertise: Advanced proficiency with Databricks, including cluster management, notebook development, and Delta Lake features. Technical skills: Strong knowledge of SQL, NoSQL databases, and big data technologies like Apache Spark (PySpark, Scala). Programming: Proficiency in programming languages like Python. Other skills: Experience with data warehousing, data modeling, and performance tuning.