IN_Associate_Data Engineer­_Data & Analytics _Advisory _Pune

Responsibilities: 

• Design, build, and maintain scalable and reliable data pipelines for batch and real-time data processing on cloud platforms.  

• Develop, construct, test, and maintain data architectures including databases, data lakes, and large-scale processing systems primarily on cloud environments.  

• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver data solutions leveraging cloud services.  

• Optimize data workflows for performance, cost, scalability, and security using cloud-native tools and best practices.  

• Implement and maintain ETL/ELT processes using cloud-based data integration and orchestration tools.  

• Ensure data quality, integrity, and security across cloud data platforms, working with data governance teams.  

• Monitor, troubleshoot, and resolve data pipeline issues in cloud environments.  

• Document data flow architectures, metadata management, and cloud resource configurations.  

Required Skills:  

• Strong experience with SQL and NoSQL databases.  

• Proficiency in programming languages such as Python, Java, or Scala.  

• Hands-on experience with big data processing frameworks like Hadoop, Spark, or Kafka.  

• In-depth knowledge of data warehousing and ETL/ELT design patterns.  

• Extensive experience with cloud platforms and services such as AWS (e.g., S3, Redshift, Glue, Lambda), Microsoft Azure (e.g., Data Factory, Synapse, Blob Storage), or Google Cloud Platform (e.g., BigQuery, Dataflow, Pub/Sub).  

• Familiarity with cloud security, governance, and cost management best practices.  

• Strong understanding of data modeling, data governance, and data quality principles.  

• Ability to design cloud-native solutions to meet business data needs effectively.  

• Strong analytical, problem-solving, and communication skills.  

Mandatory skill sets: 

Data Engineering/Pyspark  

Preferred skill sets: 

Data Engineering/Pyspark  

Years of experience required: 

4—8 Years  

Education qualification: 

B.E.(B.Tech)/M.E/M.Tech