AWS Data Engineer

Requirements

Key Responsibilities

• Develop modules/code for migration of existing Matillion jobs to AWS-native solutions.
  
• Participate
  in Requirements Analysis of existing data processing jobs.
  
• Design,
  develop, and promote scalable data pipelines using AWS Glue and Apache
  Airflow (and other basic AWS Services) - must have.
  
• Build
  robust data ingestion, transformation, and orchestration frameworks on
  AWS.
  
• Create
  reusable and modular ETL/ELT components for enterprise data platforms.
  
• Data
  modeling and warehousing concepts – must have
  
• Data
  migration and modernization projects – good to have
  
• Implement
  data quality, monitoring, alerting, security, governance, and compliance
  standards.
  
• Collaborate
  with data architects and business teams to understand migration
  requirements and translate them into technical solutions.
  
• Optimize
  data processing performance, cost, and scalability across AWS services.
  
• Participate
  in CI/CD best practices and DevOps processes for data engineering
  workflows.
  

Mandatory Skills

Core Technologies

• Apache
  Airflow (DAG development, workflow orchestration, scheduling,
  monitoring)
  
• AWS
  Glue (ETL jobs, Glue Studio, Crawlers, Data Catalog, Spark-based
  transformations)
  
• Snowflake (Data Warehousing, SnowSQL, Snowpipe, Streams & Tasks, Performance
  Tuning)
  

• AWS Lambda
  and Step Functions (preferred)
  
• Git
  experience (must-have)
  

Programming languages: Python,
SQL, Spark (any 1 is must)

PySpark, Matillion, Shell
scripting (good to have)

Experience Requirements

• 2+
  years of intensive hands-on experience with AWS data services.
  
• Proven
  experience migrating ETL platforms such as Matillion (or other ETL tools like DataStage) to AWS-native architectures (Good to have).
  
• Strong
  experience developing workflows in Apache Airflow.
  
• Extensive
  experience building and optimizing AWS Glue jobs.
  
• Hands-on
  experience with Snowflake stored procedures (must-have), performance tuning (good-to-have).
  
• Strong
  understanding of cloud architecture, scalability, deployments.