Sr Data Engineer - Databricks

Location- Remote (Preferred candidates to be in bangalore)

• Design, implement, and optimize scalable data pipelines using Databricks and Apache Spark.
  
  


• Architect data lakes using Delta Lake, ensuring reliable and efficient data storage.
  
  


• Manage metadata, security, and lineage through Unity Catalog for governance and compliance.
  
  


• Ingest and process streaming data using Apache Kafka and real-time frameworks.
  
  


• Collaborate with ML engineers and data scientists on LLM-based AI/GenAI project pipelines.
  
  


• Apply CI/CD and DevOps practices to automate data workflows and deployments (e.g., with GitHub Actions, Jenkins, Terraform).
  
  


• Optimize query performance and data transformations using advanced SQL.
  
  


• Implement and uphold data governance, quality, and access control policies.
  
  


• Support production data pipelines and respond to issues and performance bottlenecks.
  
  


• Contribute to architectural decisions around data strategy and platform scalability.
  

Requirements