Data Streaming Engineer Apache Flink & Spark

We are looking for an experienced Data Streaming Engineer with strong hands-on expertise in Apache Flink and Apache Spark to design, develop, and optimize large-scale real-time and batch data processing solutions.

The candidate should have strong experience in Flink DataStream API, Flink Table API/SQL, Spark Core, Spark SQL, and Spark Structured Streaming, with a solid understanding of distributed stream processing and high-volume data pipelines.

Mandatory Skills

• 8–10 years of experience in Data Engineering / Big Data / Streaming.


• Strong hands-on experience with Apache Flink.


• Expertise in Flink DataStream API.


• Strong knowledge of Flink Table API & Flink SQL API.


• Experience with stateful stream processing, event-time processing, windowing, watermarks, and late-event handling.


• Strong understanding of Flink checkpointing, fault tolerance, and distributed deployment.


• Strong experience with Apache Spark.


• Hands-on expertise in Spark Core, Spark SQL, and Spark Structured Streaming.


• Experience developing high-volume ETL/data transformation pipelines.


• Strong understanding of Spark execution architecture, resource management, performance tuning, and optimization.


• Strong understanding of distributed systems and real-time data processing.


• Proficiency in at least one programming language: Java / Scala / Python.

Good to Have

• Apache Kafka / Azure Event Hubs or other messaging platforms.


• Azure / AWS / GCP cloud experience.


• Docker / Kubernetes.


• CI/CD and DevOps practices.


• Experience with Data Lakes / Lakehouse architectures.


• Microservices and event-driven architecture.


• Monitoring, logging, troubleshooting, and data quality.

Key Responsibilities

• Design and develop scalable real-time streaming pipelines using Flink and Spark.


• Build event-driven and near-real-time data processing solutions.


• Develop batch and streaming ETL workflows.


• Optimize pipelines for low latency, high throughput, reliability, and scalability.


• Implement Flink state management, checkpointing, watermarks, and fault tolerance.


• Tune Spark jobs and optimize resource utilization.


• Troubleshoot production data processing and streaming issues.


• Implement monitoring, logging, data quality, governance, and security standards.


• Collaborate with Data Engineers, Architects, and business stakeholders.