Senior Data Engineer

Job Title: Senior Data Engineer

Experience: 7–10 Years

Location: Pune / Bangalore / Hybrid (as applicable)

Employment Type: Full‑time

Job Summary

We are seeking an experienced Senior Data Engineer with strong hands‑on expertise in PySpark and Azure Data Factory (ADF) to design, build, and optimize scalable data pipelines. The ideal candidate will work closely with business stakeholders, data architects, and analytics teams to deliver high‑quality, reliable, and performant data solutions on cloud‑based platforms.

• Design, develop, and maintain end‑to‑end data pipelines using PySpark for large‑scale data processing.
• Build and orchestrate data ingestion, transformation, and loading workflows using Azure Data Factory (ADF).
• Implement batch and incremental data processing patterns for structured and semi‑structured data.
• Optimize Spark jobs for performance, scalability, and cost efficiency.
• Develop reusable data frameworks and follow best practices for coding, logging, and error handling.
• Collaborate with data architects to implement data models (conceptual, logical, and physical).
• Ensure data quality, validation, and reconciliation across source and target systems.
• Work with Azure services such as Azure Data Lake Storage (ADLS), Azure SQL, Synapse.

• 7–10 years of experience in data engineering or related roles.
• Strong hands‑on experience with PySpark ,Azure Synapse and Spark SQL.
• Proven experience building pipelines using Azure Data Factory (ADF).
• Solid understanding of ETL/ELT concepts, data warehousing, and data lakes.
• Strong SQL skills for data analysis and transformations.
• Experience working with Azure cloud ecosystem, especially ADLS Gen2.
• Knowledge of data partitioning, file formats (Parquet, ORC, Avro), and performance tuning.
• Experience with source systems such as RDBMS, flat files, APIs, or streaming data (nice to have).