Data Engineer (PySpark and Apache Airflow)

About
Aligned Automation

At
Aligned Automation, we live by our "Better
Together" philosophy to build a better world. As a strategic service
provider to Fortune 500 companies, we help digitize enterprise operations and
drive impactful business strategies. Our purpose goes beyond projects—we strive
to deliver meaningful, sustainable change that shapes a more optimistic and
equitable future.

Our
culture is deeply rooted in our 4Cs—Care, Courage, Curiosity,
and Collaboration—ensuring that each employee is empowered to grow,
innovate, and thrive in an inclusive workplace.

We are seeking a skilled Data Engineer with strong
expertise in PySpark and Apache Airflow to design, build, and
optimize scalable data pipelines. The ideal candidate should have experience in
big data processing, workflow orchestration, and cloud -based data platforms.

Key Responsibilities

• Design,
  develop, and maintain scalable ETL/ELT pipelines using PySpark
  
• Build
  and manage workflow orchestration using Apache Airflow
  
• Process
  large datasets using distributed computing frameworks (Spark)
  
• Optimize
  data pipelines for performance, reliability, and scalability
  
• Implement
  data quality checks and monitoring mechanisms
  
• Work
  closely with Data Analysts, Data Scientists, and BI teams
  
• Manage
  data ingestion from various sources (APIs, databases, flat files,
  streaming)
  
• Troubleshoot
  and resolve pipeline failures
  
• Implement
  CI/CD for data pipelines
  
• Ensure
  data governance and security best practices
  

Required Skills

Technical Skills:

• Strong
  hands -on experience in PySpark
  
• Experience
  in Apache Airflow (DAGs, Operators, Scheduling)
  
• Good
  understanding of Spark architecture
  
• Strong
  SQL knowledge
  
• Experience
  with data warehousing concepts
  
• Experience
  with:
  
• S3
  / ADLS / GCS
  
• Redshift
  / Snowflake / BigQuery
  
• Knowledge
  of Git and version control
  
• Understanding
  of REST APIs and data ingestion
  

Good to Have:

• Experience
  with cloud platforms (AWS / Azure / GCP)
  
• Experience
  with Kafka or streaming pipelines
  
• Docker
  & Kubernetes knowledge
  
• Delta
  Lake / Iceberg knowledge
  
• Experience
  in CI/CD tools (Jenkins, GitHub Actions)
  
• Experience
  in monitoring tools (Prometheus, Grafana)
  

Educational
Qualification

• Bachelor’s
  or Master’s degree in Computer Science, IT, Engineering, or related field
  

Soft Skills

• Strong
  problem -solving skills
  
• Good
  communication and collaboration skills
  
• Ability
  to work in an agile environment
  
• Ownership
  mindset and attention to detail