Sr. Data Engineer

This role is for one of the Weekday's clients

Min Experience: 5 years

Location: Bengaluru, NCR, Delhi

JobType: full-time

Key Responsibilities

• Design, develop, and optimize scalable data pipelines using PySpark and SQL.
• Build and maintain ETL/ELT workflows for ingesting, transforming, and loading data from multiple sources.
• Develop data solutions using the Databricks platform.
• Work with cloud services, preferably Microsoft Azure, to implement data engineering solutions.
• Optimize data processing performance and ensure data quality, reliability, and governance.
• Collaborate with data analysts, data scientists, and business teams to deliver high-quality data products.
• Implement best practices for data modeling, data pipeline orchestration, and performance tuning.
• Monitor, troubleshoot, and improve existing data pipelines and workflows.
   

Required Skills

• Strong hands-on experience with PySpark and SQL.
• Experience developing ETL/ELT pipelines for large-scale data processing.
• Hands-on experience with Databricks.
• Experience with cloud platforms, preferably Microsoft Azure.
• Knowledge of distributed data processing frameworks such as Apache Spark.
• Experience working with data lakes, data warehouses, and big data ecosystems.
• Understanding of data pipeline orchestration tools.
• Strong problem-solving and performance optimization skills.
   

Preferred Qualifications

• Experience with Azure Data Factory or similar orchestration tools.
• Knowledge of Delta Lake architecture.
• Experience with CI/CD and version control tools like Git.
• Familiarity with data governance and data quality frameworks.

Skills

PySpark

SQL

ETL

Databricks