Databricks - Workstream Lead

• Jobseeker Video Testimonials 
  
• Employee Glassdoor Reviews
  

We are an IT Solutions Integrator/Consulting Firm helping our clients hire the right professional for an exciting long -term project. Here are a few details.

Requirements

We are seeking an experienced Offshore Delivery Lead to oversee and manage the end -to -end technical delivery of large -scale data engineering solutions. This role requires strong hands -on expertise in PySpark, Apache Spark, AWS, and Databricks, along with proven leadership in managing offshore delivery teams.

The ideal candidate will act as a technical leader, delivery owner, and client -facing coordinator, ensuring scalable, high -performance data solutions are delivered on time and within scope, while maintaining quality and engineering best practices.

Key Responsibilities

Delivery & Program Management

• Lead offshore data engineering teams and oversee end -to -end technical delivery.
  

• Own sprint planning, delivery milestones, and execution in Agile frameworks.
  

• Ensure on -time, high -quality delivery aligned with client expectations and SLAs.
  

• Identify delivery risks and proactively implement mitigation strategies.
  

Technical Leadership

• Provide hands -on technical guidance in PySpark, Spark, AWS, and Databricks.
  

• Review solution designs, code, and implementations for performance, scalability, and reliability.
  

• Define and enforce data engineering standards, coding guidelines, and best practices.
  

• Drive reusable frameworks, performance optimization, and cloud -native designs.
  

Architecture & Solution Design

• Design and guide implementation of scalable data pipelines and data platforms.
  

• Architect ETL/ELT workflows using PySpark and Spark on Databricks.
  

• Ensure optimal use of AWS services such as:
  

  • Amazon S3, EMR, Glue, Lambda
    

  • Redshift, Aurora, Athena
    

• Support batch and streaming architectures where applicable.
  

Stakeholder & Client Engagement

• Act as the primary offshore point of contact for onsite leads and client stakeholders.
  

• Translate business and functional requirements into clear technical deliverables.
  

• Provide regular delivery status updates, metrics, and risk reports.
  

• Collaborate closely with onsite teams for seamless execution.
  

Quality Assurance & Governance

• Ensure adherence to data security, compliance, and governance standards.
  

• Drive automated testing, data validation, and monitoring strategies.
  

• Ensure robustness of production deployments and smooth handovers to support teams.
  

Team Leadership & Mentoring

• Mentor and coach data engineers, helping them grow technically and professionally.
  

• Conduct technical reviews, performance feedback, and knowledge -sharing sessions.
  

• Build a high -performing, scalable offshore delivery team.
  

Required Skills & Qualifications

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
  

• 8+ years of overall experience in data engineering / big data solutions.
  

• 3+ years of experience leading offshore delivery teams.
  

• Strong hands -on expertise in:
  

  • PySpark and Apache Spark
    

  • Databricks (workspace management, jobs, notebooks, Delta Lake)
    

  • AWS cloud services
    

• Strong experience designing and building ETL/ELT pipelines.
  

• Solid understanding of data modeling, partitioning, and performance tuning.
  

• Proven experience working in Agile / Scrum delivery models.
  

• Excellent communication, leadership, and stakeholder management skills.
  

Nice -to -Have Skills

• Experience with streaming technologies (Kafka, Spark Streaming, Kinesis).
  

• Exposure to data governance, data quality, and metadata tools.
  

• CI/CD pipeline experience for data platforms.
  

• Knowledge of orchestration tools such as Airflow or AWS Step Functions.
  

• Prior experience working with global or onsite -offshore delivery models.
  

• Experience in domains such as finance, insurance, healthcare, or retail.
  

Benefits