Senior Data Engineer (GenAI / AWS)

Our client is seeking a Senior Data Engineer with 6-10 years of experience to design and build advanced data pipelines and AI-powered document processing solutions on AWS. This role focuses on developing end-to-end workflows that integrate generative AI, vector databases, and RAG architectures to support intelligent agent systems.

Responsibilities & Qualifications

• Design and develop scalable data pipelines using AWS Step Functions, Lambda, S3, Textract, and Comprehend for extracting and processing structured and unstructured data
• Build automated document-processing workflows with document chunking, embeddings, vector indexing, and semantic search capabilities
• Develop and maintain vector databases and knowledge bases to support RAG (Retrieval-Augmented Generation) systems and integrate them with AI agents for contextual information retrieval
• Build and deploy AI agents using Claude Agent SDK and Amazon Bedrock AgentCore Runtime
• Create and maintain CI/CD pipelines using GitHub Actions to automate builds, testing, and deployments
• Optimize data pipelines for scalability, reliability, security, and performance across production environments
• Implement infrastructure-as-code solutions using Terraform and AWS CDK for repeatable, maintainable deployments

Requirements

• 6-10 years of professional experience as a Data Engineer or similar role
• Strong proficiency in Python, SQL, and TypeScript
• Hands-on experience with AWS services: Lambda, S3, Step Functions, Glue, IAM, and Bedrock
• Demonstrated expertise with vector databases (Pinecone, pgvector, OpenSearch) and embedding technologies
• Experience building RAG systems, knowledge bases, and semantic search solutions
• Proficiency with AI/ML frameworks and libraries: LangChain, LlamaIndex, and Claude Agent SDK
• Experience with Document AI, OCR technologies (Textract), and NLP services (Comprehend)
• Familiarity with infrastructure-as-code tools (Terraform, AWS CDK) and CI/CD automation (GitHub Actions)