Azure DataBricks_5 years

We are looking for a skilled Data Engineer with strong hands-on experience in Azure and Databricks to build scalable data pipelines and optimize data workflows. The ideal candidate will be responsible for designing ETL/ELT processes, transforming and enriching data, handling structured and unstructured datasets, and supporting business analytics through efficient data modeling and performance tuning.
*MUST HAVE AZURE CERTIFICATION*
Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using Azure Databricks


• Extract, transform, and load data from multiple data sources (structured & unstructured)


• Clean, validate, and enrich data at various processing stages


• Develop and optimize data models (dimensional modeling, star schema, etc.)


• Monitor, troubleshoot, and improve performance of data pipelines and SQL queries


• Ensure data quality, consistency, and governance standards


• Collaborate with BI, analytics, and business stakeholders to understand requirements


• Implement best practices for performance tuning and cost optimization in Azure

Required Skills & Qualifications

• Strong hands-on experience with Azure Databricks


• Good understanding of Azure Data Lake, Azure Data Factory, and related Azure services


• Strong SQL skills (complex joins, aggregations, query optimization)


• Experience with PySpark / Apache Spark


• Experience building and maintaining ETL/ELT workflows


• Knowledge of data modeling concepts


• Experience handling unstructured textual data


• Strong analytical and problem-solving skills

Good to Have

• Experience with Delta Lake


• CI/CD experience for data pipelines


• Knowledge of data governance and security best practices


• Exposure to cloud data warehousing solutions