AI Knowledge Engineer

Be a Part of Something BIG! 

• Design, build, productionize, and support scalable document parsing and content extraction pipelines on a modern hybrid cloud platform, handling structured and unstructured data including PDFs, PPTX, Excel, Word, images, scanned documents, videos, audios, charts, tables etc
• Design and implement OCR, VLM, and multimodal AI pipelines leveraging state-of-the-art models (e.g., GPT- [4/5]o, vision-language models, Text-to-Speech) for accurate content extraction, layout understanding, and table/chart interpretation
• Develop and optimize embedding models, including fine-tuning and domain adaptation to support organization-specific terminology, keywords, and semantic search use cases
• Implement knowledge ingestion, indexing, and retrieval pipelines for RAG (Retrieval-Augmented Generation) and agentic GenAI systems, ensuring high recall, precision, and low-latency retrieval

Make an Impact by:

• Build and maintain data ingestion pipelines for batch and streaming data sources using tools/library like Kafka, LlamaIndex, LangChain, PyMuPDF, MSAL etc.
• Implement layout-aware parsing to accurately preserve document structure, hierarchy, and reading order (using OCR, VLM, and multimodal AI solutions).
• Extract accurate numeric and semantic data from complex charts and irregular tables while preventing hallucinations.
• Assist in integrating data from diverse source systems (files, APIs, databases, streaming)
• Implement, fine-tune, and optimize embedding models to support organization-specific terminology, improving semantic search, retrieval accuracy, and downstream GenAI performance.
• Help maintain metadata and pipeline documentation for transparency and traceability
• Participate in integrating pipelines with tools such as Microsoft Fabric, Databricks, Delta Lake, and other platform components
• Build and maintain knowledge base and RAG solution on cloud (Azure)
• Implement and operate knowledge base storage, lifecycle management and embedding/vectorization
• Contribute to automation efforts using version control and CI/CD workflows
• Apply basic data governance and access control policies during implementation.

Skills for Success: 

• Bachelor’s degree in Computer Science, Engineering, or a related field
• 1–3 years of experience in data engineering or data platform development. Fresh graduates encouraged to apply as well.
• Proven ability to independently build basic batch or streaming document processing pipelines
• Hands-on experience with Python, PyTorch, PEFT, LoRA, OCR, VLM, LLM, prompting, Text-to-Speech models and SQL for data transformation and validation
• Familiarity with Apache Spark (especially PySpark) and large-scale data processing concepts
• Experience with implementing knowledge base and RAG solutions for agentic AI use cases
• Self-starter with strong problem-solving skills and a keen attention to detail
• Able to work independently while collaborating effectively with senior engineers and other stakeholders
• Strong documentation and communication skills