Senior Data Engineer

Role overview:

You will be joining one of Australia's largest financial institutions, within the fastest paced financial markets area of the business. Working as part of a new team of highly skilled engineers, you will be working on building a new regulatory audit response capability, building controls around validating rates in trading.

As a Senior Data Engineer, you will play a crucial role in developing and maintaining the data infrastructure, ensuring the seamless integration and analysis of vast amounts of data. Your expertise will directly contribute to market-leading strategies and decision-making processes.

Day to day you will...

• Develop and maintain scalable, reliable, and secure data pipelines and architectures.
• Design, implement, and optimize complex SQL queries and Data Definition Language (DDL) operations.
• Utilize your strong hands-on coding ability in Python to build robust data solutions.
• Deploy and manage applications in a clustered environment using Docker and Kubernetes.
• Leverage AWS services to enhance our data processing and storage capabilities.
• Ensure seamless integration with modern systems through API-based solutions.
• Query and analyze big data using tools such as Redshift, Hive, Spark, and Presto.
• Implement and manage data pipeline orchestration tools like Airflow or Argo Workflows.
• Utilize source control tools like Bitbucket to manage and collaborate on codebase effectively.
• Apply your knowledge of Linux/Unix systems to maintain and optimize our data infrastructure.

We are looking for...

• 7+ years of hands on Python coding experience working on complex applications
• Experience querying big data with tools like Redshift, Hive, Spark, and Presto, orchestrating data pipelines with Airflow or Argo Workflows
• Strong SQL experience, authoring complex queries and Data Definition Language (DDL)
• Experience within Linux/Unix environments
• Experience working in cloud environments, ideally AWS
• Experience with CI/CD tools
• Working knowledge of DevOps principals and best practices

** AU PR or Citizenship candidates only**

** Sydney based candidates only, Hybrid model (3 days a week in CBD office) **

Sound like an interesting next move, and have the skills to suit? Apply now!