AWS Data Engineer - Contract - Sydney

Requirements

• Design and implement scalable data pipelines using Python or PySpark, and boto3
  


• Integrate with Amazon Redshift for data warehousing and perform data ingestion and transformation tasks
  


• Work with Amazon S3 to architect and manage data lakes
  


• Build and manage batch processing jobs using AWS Glue, EMR, and Lambda
  


• Support streaming data workflows using Amazon Kinesis
  


• Leverage AWS Secrets Manager and KMS for secure data handling and encryption
  


• Write efficient SQL for data querying, transformations, and performance tuning
  


• Collaborate with data engineering, product, and DevOps teams to deliver end-to-end solutions
  


• Participate in code reviews, debugging, and testing activities to ensure high-quality software delivery
  


• Use Git for version control and follow best practices in CI/CD
  


• Continuously explore and recommend improvements to enhance system performance, scalability, and maintainability
  

• Advanced proficiency in Python, including libraries such as Python/ PySpark , Pandas, , and boto3
  


• Experience designing workflows in Apache Airflow (or MWAA)
  


• Hands-on experience with Amazon Redshift for analytics and data warehousing
  


• Solid SQL skills and experience with data modeling and transformation
  


• Comfortable with Amazon S3, including setting up data lake architecture
  


• Familiar with AWS core services: EC2, S3, IAM, Lambda
  


• Experience using AWS Secrets Manager and AWS KMS for secure data access
  


• Strong understanding of batch processing (AWS Glue, EMR) and streaming (Kinesis)
  


• Version control using Git
  


• Excellent problem-solving and communication skills
  

• Knowledge of additional ETL/ELT tools like AWS Glue, AWS Athena
  


• Familiarity with DevOps practices and CI/CD pipelines
  


• Experience working in agile teams and data-focused environments
  


• Knowledge in read and understand the JavaScript (Node.js/Express)