Responsibilities:
- Collaborate with domain experts to understand business needs and translate them into clear and effective LLM prompts.
- Develop, test, and refine prompts for a variety of use cases, ensuring high accuracy and relevance in generated outputs.
- Troubleshoot and resolve prompt-related issues and bugs in a timely manner.
- Design multi-step prompt workflows, including agentic patterns, tool calling, and retrieval-augmented generation (RAG) setups.
- Work closely with software developers to integrate LLM solutions into Product.
- Define and maintain qualitative and quantitative evaluation metrics for LLM outputs (e.g., accuracy, relevance, safety, latency, cost).
- Contribute to the development of a scalable framework for prompt engineering, promoting reuse and knowledge sharing across different business units.
- Assist in gathering and analyzing user feedback to continually improve prompt effectiveness.
- Stay up-to-date with advancements in LLM technology and contribute to strategic decisions on tool and platform selection.
- Document prompt engineering processes, best practices, and use case outcomes to build a comprehensive knowledge base.
Requirements:
- Bachelor's degree in Computer Science, Engineering, Mathematics, or related field with 4+ years of experience in AI/ML, NLP, or applied LLM systems.
- Hands-on experience with LLM-powered systems, including RAG, embeddings, function/tool calling, and orchestration frameworks.
- Familiarity with tools such as OpenAI's GPT models, Hugging Face, etc.
- Experience evaluating LLM outputs and debugging failure modes such as hallucinations, prompt leakage, and edge cases.
- Proficiency in Python or other programming languages commonly used in AI/ML development.
- Strong problem-solving skills and attention to detail.
- Excellent communication skills, with the ability to articulate complex technical concepts to non-technical stakeholders.
- Ability to work collaboratively in a team environment and adapt to changing project requirements.
- English proficiency - upper-intermediate level or higher.