Senior Backend Engineer Distributed Systems (Kafka, Data Ingestion)

About the Role

We are seeking a Software Engineer to design and build high‑performance distributed data ingestion systems. You will work on parallel processing pipelines that integrate with cloud storage, relational databases, and streaming platforms. This is a core product development role where you’ll apply best practices in distributed system design, ensuring scalability, reliability, and performance.

Key Responsibilities

• Design and implement data ingestion pipelines with parallel processing in Java, C++, or Golang.
  
• Ingest data from Amazon S3, Azure Cloud Storage, Google Cloud Storage, Snowflake, BigQuery, PostgreSQL, files, Kafka, and Iceberg data lakehouse.
  
• Build high‑availability (HA) loading systems with cross‑region replication.
  
• Develop monitoring and error reporting for ingestion pipelines.
  
• Integrate with Spark connectors and manage third‑party systems (Kafka, Kafka Connect).
  
• Collaborate with cross‑functional teams to ensure scalable, reliable, and performant distributed systems.
  
• Follow Agile development practices and contribute to CI/CD pipelines.
  

Requirements

Requirements

• Strong programming skills in Java, C++, or Golang.
  
• Hands‑on experience with Kafka, Zookeeper, Spark, or stream processing frameworks.
  
• Expertise in Kafka Connect, Kafka Streams, Kafka security, and customization.
  
• Experience with Spark connectors and event‑driven architectures.
  
• Familiarity with Agile development and CI/CD workflows.
  

Nice to Have

• Experience with gRPC protocol and multi‑threading.
  
• Exposure to Zookeeper, ETCD, or Consul.
  
• Understanding of distributed consensus algorithms (Paxos/Raft).
  
• Knowledge of Docker and Kubernetes for containerized deployments.