Data Engineer (Databricks + AI)

Job Summary

We are looking for an experienced AI / Data Engineer to design and build scalable data pipelines and AI-powered applications. The ideal candidate will combine strong data engineering fundamentals with hands-on experience in LLM frameworks, Retrieval-Augmented Generation (RAG), and API development to deliver production-grade AI and data solutions.

Key Responsibilities :

• Design, develop, and optimize scalable data pipelines using Azure/AWS/GCP Databricks and PySpark.


• Develop AI-powered applications using LLMs, LangChain, and LlamaIndex.


• Design and implement RAG (Retrieval-Augmented Generation) solutions using enterprise knowledge bases.


• Develop REST APIs using Python (FastAPI/Flask) for AI and data services.


• Collaborate with Data Scientists, ML Engineers, and Business stakeholders to deliver AI-driven solutions.


• Optimize model performance, prompt engineering, and inference pipelines.


• Follow DevOps and MLOps best practices for deployment, monitoring, and maintenance.

Required Skills & Qualifications :

• 3+ years of experience in data engineering, with strong expertise in AWS/Azure/GCP Databricks and PySpark/Python.


• Proven experience building and maintaining ETL/ELT pipelines at scale.


• Hands-on experience with LLM frameworks such as LangChain and LlamaIndex.


• Practical knowledge of RAG architecture and enterprise knowledge base integration.


• Strong Python skills, including API development using FastAPI or Flask.


• Familiarity with prompt engineering and inference pipeline optimization.


• Understanding of DevOps/MLOps practices (CI/CD, model deployment, monitoring).


• Strong communication and collaboration skills to work across cross-functional teams.