Data BAU Lead

Responsibilities

We are looking for a seasoned BAU Lead - Data Engineering to lead our data

operations and support function in a cloud-first environment. The ideal

candidate will bring strong technical leadership, hands-on expertise in AWS

and Snowflake, and proven experience in managing Business-As-Usual (BAU)

data platforms, production support, and continuous improvement initiatives.

▪ Develop tools to improve data flows between internal/external systems

and the data lake/warehouse.

▪ Work with stakeholders to understand needs for data structure, availability,

scalability, and accessibility.

▪ Build robust and reproducible data ingest pipelines to collect, clean,

harmonize, merge, and consolidate data sources.

▪ Understanding existing data applications and infrastructure architecture

▪ Build and support new data feeds for various Data Management layers and

Data Lakes

▪ Evaluate business needs and requirements.

▪ Support migration of existing data transformation jobs in Oracle, and MS[1]SQL to Snowflake.

▪ Lead the migration of the existing data transformation jobs in Oracle, Hive,

Impala etc. into Spark, Python on Glue etc.

▪ Able to document the processes and steps.

▪ Develop and maintain datasets.

▪ Improve data quality and efficiency.

▪ Lead Business requirements and deliver accordingly.

▪ Collaborate with Data Scientists, Architect and Team on several Data

Analytics projects.

▪ Collaborate with DevOps Engineer to improve system deployment and

monitoring process.

▪ Experience in critical production support and how the BAU functions is

preferred.

Requirements Key Skills:

▪ Lead a team of data engineers in managing day-to-day BAU operations,

production support, incident management, and platform stability.

▪ Strong AWS knowledge in terms of designing new architecture and

providing optimized solutions for existing ones. (S3, Glue, DMS, MWAA,

AMS, IAM).

▪ In-depth knowledge with respect to Snowflake and its architecture.

▪ Prefer prior Experience in BAU environment.

▪ Good knowledge on Airflow and MWAA.

▪ Hands-on experience in SQL/Python/Pyspark.

▪ Expertise in optimizing techniques in cloud environments.

▪ Should have the vision on data strategy and able to deliver the same.

▪ Should be able to lead the design and implementation of data management

processes, including data sourcing, integration, and transformation.

▪ Able to manage and lead a team of data professionals, providing guidance,

mentoring and foster a collaborative and innovative team culture focused on

continuous improvement.

▪ To evaluate and recommend data-related technologies, tools, and

platforms.

▪ Collaborate with IT teams to ensure seamless integration of data solutions.

▪ Should have experience in Implementing and enforcing data security

protocols and ensure compliance with relevant regulations.

Required Qualifications:

▪ At least 8+ years of Data Engineer Experience

▪ Bachelor qualification in a computer science or STEM (science, technology,

engineering, or mathematics) related field.

▪ At least 5+ years of recent hands-on professional experience (actively

coding) working as a Lead handling support & production issue.

▪ 2+ experience with large scale datasets, data lake and data warehouse

technologies such as AWS Redshift, Google BigQuery, Snowflake. Snowflake

is highly preferred.

▪ At least 3+ years of experience in ETL (AWS Glue), Amazon S3, Amazon

RDS, Amazon Kinesis, Amazon Lambda, Apache Airflows, Amazon Step

Functions.

▪ Professional experience working in an agile, dynamic and customer-facing

environment is required.

▪ Understanding of distributed systems and cloud technologies (AWS) is

highly preferred.

▪ Understanding of data streaming and scalable data processing is preferred

to have.

▪ Strong knowledge in scripting languages like SQL ,Python, UNIX shell and

Spark is required.

▪ Understanding of RDBMS, Data ingestions, Data flows, Data Integrations

etc.

▪ Technical expertise with data models, data mining and segmentation

techniques.

▪ Experience with full SDLC lifecycle and Lean or Agile development

methodologies.

▪ Knowledge of CI/CD and GIT Deployments.

▪ Ability to work in team in diverse/ multiple stakeholder environment.

▪ Ability to communicate complex technology solutions to diverse teams

namely, technical, business and management teams

Soft Skills

▪ Ability to work in a collaborative environment and coach other team

members on coding practices, design principles, and implementation

patterns that lead to high quality maintainable solutions.

▪ Excellent communications and stake-holder management are required.

▪ Ability to handle senior leadership and report to them if required.

▪ Ability to work in a dynamic, agile environment within a geographically

distributed team.

▪ Ability to focus on promptly addressing customer needs.

▪ Ability to work within a diverse and inclusive team.

▪ Technically curious, self-motivated, versatile and solution-oriented