Sr Data Engineer - Apache DolphinScheduler - Chennai

Key Responsibilities:

• Develop, configure, and optimize Apache DolphinScheduler workflows for data processing and automation.


• Design and implement ETL pipelines, job scheduling, and workflow orchestration solutions.


• Troubleshoot and resolve performance, scalability, and reliability issues in DolphinScheduler.


• Integrate DolphinScheduler with big data tools such as Hadoop, Spark, Flink, Hive, and Kafka.


• Work closely with data engineers, DevOps, and software teams to enhance workflow automation.


• Develop custom plugins and extensions for DolphinScheduler as needed.


• Monitor and optimize job execution, resource allocation, and workflow efficiency.


• Maintain best practices for CI/CD, version control, and infrastructure automation.

Required Skills & Qualifications:

• Strong experience with Apache DolphinScheduler in a production environment.


• Proficiency in Java, Python, or Scala for workflow scripting and automation.


• Experience with big data technologies like Hadoop, Spark, Hive, Flink, and Kafka.


• Understanding of workflow orchestration, DAG execution, and scheduling strategies.


• Strong problem-solving skills and ability to work in a fast-paced environment.