Senior Data Engineer

Organization:
Commonwealth Bank of Australia is the largest and number one bank in Australia, serving over 17 million customers. We maintain a high-trust brand reputation built on financial strength, innovation, and community impact. As of July 2025, CBA holds market cap of approximately USD 195.7 billion, ranking us among the top 100 most valuable companies globally. Our leadership in digital banking, customer safety, and responsible AI reflects our commitment to building a brighter future for all. Our values are reflected in our high-trust brand reputation. CBA is consistently ranked among Australia’s most valuable and trusted brands.  

Job Title: Senior Data Engineer
Location: Bangalore - Manyata Tech Park

Business & Team: Retail Tech

Impact & Contribution:

• Passionate about building next generation cloud data platforms (Snowflake, AWS) and data pipeline solutions across the bank.

• Enthusiastic and able to contribute to, and learn from, the wider engineering talent in the team.

• Ready to execute state-of-the-art coding practices, driving high-quality outcomes to solve core business objectives and minimise risks.

• Capable of creating both technology blueprints and engineering roadmaps for a multi-year data transformation journey.

• Can lead and drive a culture where quality, excellence and openness are championed.

• Constantly thinking outside the box and breaking boundaries to solve complex data problems.

Roles and Responsibilities

• Contribute to the full development life cycle, including requirement analysis, functional design, technical design, programming, testing, documentation, implementation and ongoing technical support for agile projects.

• Contribute to projects/CRs utilising Snowflake, DBT, Python, PySpark, AWS cloud technologies, Ab-Initio or any ETL tools, Teradata and Oracle.

• Understand our customers' core business objectives and build end-to-end data-centric solutions to address them.

• Lead or participate in the definition, execution and communication of ETL strategy and technical solution.
• Develop complex data transformations, data cleansing and data aggregation logic using Ab-Initio or any other ETL tools, DBT, PySpark and SQL.

• Ensure high-quality code by performing thorough testing, code reviews and debugging pipelines, internal framework and components, dbt models and PySpark jobs.

• Cooperate with business and operational users, other data management staff and infrastructure teams to ensure adherence to client and company guidelines, restrictions and requirements as well as industry standards.

• Improve team development related processes that accelerate delivery, drive innovation, lower cost and improve quality.

• Adhere to the Code of Conduct. The Code of Conduct sets the standards of behaviour, actions and decisions we expect from our people. 

Must have Skills

• Snowflake: Hands-on experience designing and building data solutions on Snowflake, including virtual warehouses, streams, tasks, Snowpipe, time travel and performance optimisation. 

• dbt: Strong experience building modular, tested and documented data transformation models using dbt, including incremental models and CI/CD-driven workflows.

• AWS: Hands-on experience building cloud data solutions on AWS, including S3, Glue, EMR / EMR Serverless, Lambda, Step Functions / MWAA (Airflow), EventBridge, SQS, Kinesis, MSK/Kafka, Redshift, Athena, Iceberg and IAM. AWS certification (e.g. AWS Certified Data Engineer) preferred.

• PySpark: Proven experience developing scalable, distributed data processing pipelines using PySpark.

• SQL: Advanced proficiency in writing complex, performant SQL for data transformation, analysis and pipeline development.

• Teradata: Solid experience with Teradata data warehousing, including utilities, query performance tuning and data modelling.

• ETL tools: Preferably extensive hands-on for enterprise ETL development (Ab-Intio, DataStage and etc.)

Essential Skills:

In addition to the mandatory skills above, we use a broad range of tools, languages, and frameworks. We don’t expect you to know them all but experience or exposure to some of these (or equivalents) will set you up for success in this team.

• 8+ years of experience in design or development in Analytics and Data Warehousing projects.

• Experience using AI tools such as Claude/Roo/Cline 

• Experience in designing, building, and delivering enterprise-wide data ingestion, data integration and data pipeline solutions using Big Data and Data Warehouse platforms (e.g. Teradata, Snowflake). Preferably with at least 8 years of hands-on experience in a Data Engineering role.

• Extensive experience in ETL tools like GDE, Express-IT, Control centre, Mhub, Conduct-IT and Meta programming.

• Performance Optimisation: Optimising across environments, systems and resources.

• Root Cause Analysis: Debugging complex data issues across systems.

• Data Quality Frameworks: Implementing validation, cleansing, and reconciliation logic.

• Experience in building data solutions using AWS Cloud technology (EMR, Glue, Iceberg, Kinesis, MSK/Kafka, Redshift, S3, etc.). Preferably with 3+ years of hands-on experience and an AWS Certified Data Engineer certification.

• Strong Unix/Linux Shell scripting and programming skills.

• Proficient in SQL scripting, writing complex SQLs for building data pipelines on Teradata, Snowflake and Oracle.

• Experience in leading and mentoring data engineers, including ownership of internal business stakeholder relationships and working with consultants.

• Experience in working in Agile teams, including working closely with internal business stakeholders.

• Experience in development, production support and troubleshooting issues.

• Experience in process optimisation (e.g. SQL, ETL).

• Experience with Big Data platforms for ETL development on the AWS cloud platform, using PySpark on EMR and Glue.

• Should have experience with Teradata and Oracle SQL database programming, SQL performance tuning, and relational model analysis.

• Good automation skills in areas such as data ingestion, APIs, file transfers.

• Knowledge and experience with Dev SecOps practices and tools, including CI/CD for dbt models and data pipelines.

Education/Qualifications

    Bachelor’s degree in engineering in Computer Science/Information Technology