Senior Data Engineer

What is expected of you at this level :
We are seeking an experienced Senior Data Engineer with 5+ years in data engineering to join our team. In this role, you will design, build, and optimize high-performance data pipelines on Azure, delivering scalable, efficient, and reliable data solutions. You will take ownership of developing and maintaining critical data workflows, ensuring they meet performance and quality standards. This position requires a strong focus on performance tuning, code optimization, deployment automation, and advanced debugging, as well as the ability to work independently and collaboratively to solve complex data challenges and drive continuous improvements across our data engineering practices.

What you must have :
5+ years of data engineering experience, building, and managing large-scale data pipelines

• Proven experience working with Azure Databricks and PySpark/Spark for data engineering tasks, where one has,


• Demonstrated expertise in advanced data engineering techniques and a track record of delivering complex data engineering projects.


• Ability to independently code, troubleshoot, and test complex tasks using appropriate technologies.


• Experience with data modeling concepts related to reporting and analytics, including dimensional modeling and star/snowflake schemas.


• Experience leveraging AI-assisted development tools (such as GitHub Copilot, ChatGPT, Claude or similar) to enhance productivity, code quality, and problem-solving in data engineering tasks.
  
  

Strong proficiency in following skills,

• Python, PySpark, Spark, Databricks, Delta Lake, and SQL with hands-on experience using Azure Data Factory, Azure


• Pipelines, Azure DevOps, and Git for version control.


• Solid understanding of data modeling and working with complex data types (arrays, structs, maps).


• In-depth knowledge of performance optimization techniques for large-scale data processing, including code tuning and resource .


• Proficiency in developing, maintaining, and enhancing CI/CD processes, including documentation and monitoring of data pipelines and systems.

Leadership & Communication Skills,

• Confident, energetic self-starter with excellent interpersonal skills.


• Flexibility and adaptability to work with different technologies, platforms, and evolving requirements.


• Experience mentoring junior team members and fostering a collaborative team environment.


• Strong communication skills and ability to communicate technical concepts effectively to both technical and non-technical stakeholders.


• Demonstrates sound judgment, a sense of urgency, and a strong commitment to ethics, regulatory compliance, and business integrity.

Nice to have:

•  Understanding of DAX (Data Analysis Expressions), particularly in the context of Power BI or similar business intelligence tools.


• Exposure to business intelligence or analytics projects, bridging data engineering with reporting and visualization solutions.

Educational Requirements:

• Bachelor’s or master’s degree in computer science, information systems, engineering, or a related field.