Lead/ Sr. Pyspark Data Engineer

Company : Dataeconomy

Job Type : Full Time

Pune City, Maharashtra

Number of Applicants

:

000+

Apply Now

Let AI Supercharge Your Job Hunt!

JobCopilot scans 500,000+ company career sites daily to find jobs for you

Never miss an opportunity Save hours by auto-filling applications forms Land more interviews with tailored applications

Activate JobCopilot

Job Description - Lead/ Sr. Pyspark Data Engineer

Job Title: PySpark Data Engineer
Experience: 6+ Years
Location: Hyderabad/ Pune
Employment Type: Full -Time

Job Summary:

We are looking for a skilled and experienced PySpark Data Engineer to join our growing data engineering team. The ideal candidate will have 6+ years of experience in designing and implementing data pipelines using PySpark, AWS Glue, and Apache Airflow, with strong proficiency in SQL. You will be responsible for building scalable data processing solutions, optimizing data workflows, and collaborating with cross -functional teams to deliver high -quality data assets.

Requirements

Key Responsibilities:

Design, develop, and maintain large -scale ETL pipelines using PySpark and AWS Glue.
Orchestrate and schedule data workflows using Apache Airflow.
Optimize data processing jobs for performance and cost -efficiency.
Work with large datasets from various sources, ensuring data quality and consistency.
Collaborate with Data Scientists, Analysts, and other Engineers to understand data requirements and deliver solutions.
Write efficient, reusable, and well -documented code following best practices.
Monitor data pipeline health and performance; resolve data -related issues proactively.
Participate in code reviews, architecture discussions, and performance tuning.

Requirements

6+ years of experience in data engineering roles.
Strong expertise in PySpark for distributed data processing.
Hands -on experience with AWS Glue and other AWS data services (S3, Athena, Lambda, etc.).
Experience with Apache Airflow for workflow orchestration.
Strong proficiency in SQL for data extraction, transformation, and analysis.
Familiarity with data modeling concepts and data lake/data warehouse architectures.
Experience with version control systems (e.g., Git) and CI/CD processes.
Ability to write clean, scalable, and production -grade code.

Benefits

As per company standards.

Email address

Why are you reporting this job?

I think it’s a discriminatory or offensive

I think it’s fraudulent or a scam

I think it’s trying to sell something unrelated to the job / it’s asking for money

I think it contains incorrect or broken information

Other

All Job Ads are subject to GrabJobs’s Terms of Service. We allow users to flag postings that may be in violation of those terms. Job Ads may also be flagged by GrabJobs moderation team. However, no moderation system is perfect, and flagging a posting does not ensure that it will be removed.