Lead Data Engineer (Banking)

• Own integrated delivery for 4-6 concurrent projects, balancing priorities, dependencies, capacity, risks, and production commitments, working under a scaled agile framework.
  
  
  


• Turn transformation needs into technical scope, milestones, acceptance criteria, and measurable outcomes.
  
  
  


• Provide portfolio visibility through dashboards, risk logs, dependency tracking, and escalation.
  
  
  


• Ensure releases meet change, security, testing, documentation, and production-readiness requirements.
  
  
  
  
  
  

• Direct an 8-12 person consultant team through work allocation, onboarding, delivery follow-up, and quality review.
  
  
  


• Set an accountable rhythm with backlog refinement, design and code reviews, stand-ups, demos, and retrospectives.
  
  
  


• Build capability through reusable patterns, playbooks, pairing, coaching, and knowledge transfer.
  
  
  


• Partner with vendor and bank leaders to manage capacity, capability, performance, continuity, and key-person risks.
  
  
  
  
  
  

• Guide solutions using NiFi for ingestion, Airflow for orchestration, dbt for transformation and testing, and SQL for processing and validation.
  
  
  


• Improve pipeline stability through modular design, automated tests, version control, disciplined deployment, observability, and incident follow-up.
  
  
  


• Prioritize technical debt by business criticality, operational risk, migration value, and delivery impact.
  
  
  
  
  
  

• Follow the AWS and Databricks operating model in line with architecture, security, governance, networking, and compliance standards.
  
  
  


• Lead cloud-native pipeline delivery and phased re-engineering of on-premises pipelines.
  
  
  


• Redesign and remodeling instead of lifting and shifting when it improves scale, reliability, cost transparency, reuse, lineage, or supportability.
  
  
  


• Define migration waves, readiness, parallel runs, reconciliation, cutover, rollback, and decommissioning.
  
  
  

• Align delivery with business and data owners, stewards, architects, security, infrastructure, risk, compliance, and operations.
  
  
  


• Lead cross-domain requirements and design discussions, resolving ambiguity and clarifying trade-offs.
  
  
  


• Explain decisions, delivery health, risks, and business impact to technical and executive audiences.
  
  
  

Requirements

• Experience with Unity Catalog, Delta Lake, Databricks Workflows, CI/CD, infrastructure as code, and cloud observability.
  
  
  


• Knowledge of AWS data-platform services, including S3, IAM, KMS, CloudWatch, and private networking.
  
  
  


• Experience with catalogs, metadata, lineage, data quality, and DataOps or DataSecOps.
  
  
  


• Familiarity with dimensional modeling, data vault, medallion/lakehouse, streaming, and data products.
  
  
  


• Relevant AWS, Databricks, data engineering, or agile delivery certifications.