DataOps Engineer (AWS / Databricks)

• Lead the design and implementation of the cloud data infrastructure, with a focus on AWS + Databricks.

• Define and implement the DataOps strategy for a cloud native data platform.

• Design, implement, and maintain CI/CD pipelines for data and infrastructure, using Terraform and automation best practices.
• Define implementation standards for:

• PySpark based data pipelines
  


• Repository structure and organization
  


• Deployment, versioning, and rollback strategies
  

• Implement data governance policies, including cataloging, lineage, ownership, and data layers.

• Ensure data security controls, including Access management, encryption and environment segregation

• Define and operationalize data quality practices, such as validations, SLAs, monitoring, and observability.

• Work closely with engineering teams to standardize development workflows.

• Support the migration of data from the on premise environment to the cloud, ensuring reliability and consistency.

• Act as a technical authority for architectural decisions and the evolution of the data stack.

Requirements

• 10+ years of experience in IT.
  


• MUST HAVE experience with AWS (S3, IAM, VPC, monitoring and security services), Databricks and PySpark.
  


• Highly desirable previous experience working for Financial Services / Banks.
  


• Solid experience in DataOps, Data Engineering, or Platform Engineering.
  


• Hands on experience building cloud based data platforms.
  


• Experience with Infrastructure as Code (Terraform).
  


• Experience with data pipelines and orchestration concepts.
  


• Proven experience with CI/CD for data platforms, data governance, data quality, data security
  


• Ability to define technical standards and lead end to end implementations.
  

• Experience migrating platforms from on premise to cloud.
  


• Knowledge of Lakehouse architectures.
  


• Experience in regulated environments (financial services, insurance, healthcare).
  


• Experience with data observability tools.
  


• AWS, Databricks, or Data Engineering certifications.
  


• Bachelor’s degree in Computer Science or a related field