Sr. Pyspark Data Engineer

Company : Dataeconomy

Job Type : Full Time

Hyderabad, Telangana

Job Description - Sr. Pyspark Data Engineer

Job Title: PySpark Data Engineer
Experience: 4+ Years
Location: Hyderabad

Job Summary:

We are seeking a Senior Spark Engineer to design and implement high -performance Spark execution patterns inside xFlows, supporting batch and streaming pipelines with built -in data quality, observability, and governance.

Requirements

Key Responsibilities

Design and implement Spark -based execution frameworks for xFlows pipelines.
Build reusable Spark components for:
- Readers (JDBC, Files, Kafka, CDC)
- Transformers (Join, Filter, Aggregate, Window, Union)
- Writers (Iceberg, Delta, Parquet, Snowflake)
Optimize Spark performance (partitioning, caching, shuffles, memory).
Implement Data Quality & Reconciliation execution patterns.
Handle schema evolution, CDC, watermarking, and checkpoints.
Integrate Spark jobs with EMR Serverless / Data bricks / Kubernetes.
Publish execution metrics, logs, and lineage for observability.
Work closely with platform & UI teams to support no -code execution.

Required Skills

4+ years of experience with Apache Spark
Strong expertise in PySpark (preferred) or Spark Scala
Deep understanding of Spark internals (DAGs, stages, shuffles, caching)
Experience with Iceberg / Delta Lake
Strong SQL skills
Experience with batch and streaming pipeline

Benefits

Comprehensive Medical Coverage:
Health insurance of INR 7.0 Lakhs for you and your family (up to 6 members), ensuring complete peace of mind.
Robust Protection Plans:
Group Personal Accident Insurance and Group Term Life Insurance to safeguard you and your loved ones.
Retirement Benefits:
PF and Gratuity provided as per standard government regulations.
Flexible Work Options:
Enjoy hybrid work arrangements & flexible working hours
Generous Leave Policy:
21 days of annual leave, in addition to 10 company -declared holidays.
Employee Well -being Spaces:
Access to a dedicated break -out area with round -the -clock refreshments for relaxation and rejuvenation.

Email address

Why are you reporting this job?

I think it’s a discriminatory or offensive

I think it’s fraudulent or a scam

I think it’s trying to sell something unrelated to the job / it’s asking for money

I think it contains incorrect or broken information

Other

All Job Ads are subject to GrabJobs’s Terms of Service. We allow users to flag postings that may be in violation of those terms. Job Ads may also be flagged by GrabJobs moderation team. However, no moderation system is perfect, and flagging a posting does not ensure that it will be removed.