AI MLOPS/LLMOps Engineer

What You Will Do

AI Module Integration & Inference Pipelines

• Integrate and adjust inference pipelines for NLP modules including document classification, entity extraction, de-identification (DEID), and LLM-based early trend detection


• Connect DS-coded AI modules into end-to-end production workflows via Airflow DAGs on AWS EKS


• Build and tune hybrid search pipelines combining GTE multilingual dense embeddings with GIN lexical search on Aurora PostgreSQL


• Integrate with OpenAI-based API platform for multilingual query expansion and LLM-driven trend detection

Document Processing & Parsing

• Design and maintain document preprocessing pipelines that parse deeply nested JSON structures (emails with attachments, embedded PDFs) from S3/DataLake


• Handle multilingual unstructured text (English, Spanish, Portuguese, German, Dutch, French, Italian) across 300 GB of claim notes and documents


• Build chunking strategies and metadata extraction for downstream embedding and retrieval workflows

Data Pipeline Engineering

• Author and maintain Airflow DAGs for batch processing (monthly entity refresh, trend detection, DEID pipeline)


• Manage data flow across AWS services: S3, Athena, Glue, Fargate, SQS, Step Functions


• Scale pipelines to handle 500K+ claims and hundreds of millions of text chunks

Production Deployment & Quality

• Deploy and version models using MLflow and Databricks


• Manage schema evolution and migrations using Liquibase on Aurora PostgreSQL


• Instrument pipelines with logging, monitoring, and evaluation scoring for retrieval quality