Databricks Engineer

Job Overview

We are looking for an experienced Databricks Engineer with a strong background in Informatica PowerCenter, ETL, Data Warehousing, Data Modeling, and modern Data Lake architectures.

The ideal candidate will have strong hands-on experience building and optimizing data pipelines using Databricks, PySpark, Spark SQL, and Delta Lake, along with prior experience in Informatica PowerCenter. The role requires someone who can independently manage end-to-end data engineering activities across ingestion, transformation, modeling, quality, and analytics.

Key Skills / Essential Experience

• 8+ years of experience in data engineering / large-scale data management.


• Strong hands-on experience with Databricks.


• Excellent experience with PySpark / Spark / Spark SQL.


• Strong knowledge of Delta Lake and modern Data Lake architectures.


• Extensive experience in ETL/ELT development.


• Hands-on experience with Informatica PowerCenter, including development, support, migration, and modernization.


• Strong understanding of Data Warehousing and Data Lake concepts.


• Expertise in Data Modeling, including:
  
  
  
  • Dimensional Modeling
  
  
  • Star Schema
  
  
  • Snowflake Schema
  
  
  
  


• Experience working in a Microsoft Azure Data Platform environment.


• Good SQL programming skills.


• Strong understanding of data ingestion, transformation, integration, and processing.


• Experience with data quality, governance, and performance optimization.

Key Responsibilities

• Design, develop, and optimize scalable data pipelines using Databricks and PySpark.


• Develop and maintain robust ETL/ELT workflows for data ingestion, transformation, and loading.


• Leverage Informatica PowerCenter expertise for ETL migration, modernization, integration, and support initiatives.


• Design and implement scalable Data Lake and Data Warehouse solutions.


• Work extensively with Delta Lake, Spark SQL, and PySpark.


• Develop efficient data models to support analytics and reporting requirements.


• Implement Dimensional, Star, and Snowflake schemas as required.


• Perform data transformation and optimization across large datasets.


• Ensure data pipelines meet performance, scalability, reliability, and data quality requirements.


• Implement data governance and quality best practices.


• Collaborate with business analysts, data architects, developers, and other technical stakeholders to understand requirements and deliver robust solutions.


• Troubleshoot and resolve data pipeline, ETL, and performance issues.


• Participate in Agile development methodologies, sprint planning, reviews, and daily stand-ups.


• Contribute to ETL modernization and migration from traditional platforms to modern cloud-based data platforms.