Senior Cloud Data Engineer

A cloud data engineer is responsible for designing, building, and managing data processing systems and solutions within cloud environments. This typically involves working with a variety of cloud-based technologies, such as Amazon Web Services (AWS) to ensure that the organization's data is stored securely and efficiently.

We looking for a Cloud Data Engineer that will have a strong background in computer science, software engineering, or a related field, along with experience in database management, programming (e.g., Python, Java), and cloud computing platforms.

Responsibilities

• Designing, implementing, and managing the organization's data processing & transformation systems and pipelines on the cloud.
• Performing data migration to move and migrate data across cloud regions and cloud providers.
• Monitoring and optimizing the performance of the organization's data processing & transformation systems.
• Ensure the availability and security of databases.
• Troubleshooting complex issues with the database infrastructure.
• Line up all database schemata across all different deployments (Schema merge)
• Implementing backups and disaster recovery plans.
• Working with other IT teams to integrate the organization's databases with other systems.
• Keeping up-to-date with the latest developments in cloud-based database technologies and best practices.

• Collaborating with other IT teams to integrate the organization's databases with other systems.
• Converting/migrating databases to different database types.

• A bachelor's or master's degree in computer science, software engineering, or a related field
• +8 year experience in DB admin/SQL developer position
• Experience with Data engineering technologies in Amazon AWS (Kafka, Clickhouse) (must have )
• Experience working with cloud-based database technologies, and Amazon Web Services (AWS) is required .
• Excellent Experience of AWS RDS Aurora setup and migration MySQL
• Expertise in SQL and NoSQL databases, as well as experience with data processing & transformation tools and techniques, such as but not limited to: Apache Kafka, Apache Spark, airflow, dbt. 
• Knowledge of security best practices and experience implementing them in a cloud-based environment.
• Familiarity with agile software development methodologies and experience working in a fast-paced, dynamic environment
• Good Experience with ElasticSearch (must have).
• Good Experience with Database in Memory "Redis, Memcache" (must have)
• Proficient in writing and optimizing SQL statements.
• Ability to plan resource requirements from high-level specifications.
• Experience in NDB cluster/big data
• Experience with other DBMS, such as but not limited to: MangoDB, PostgreSQL, Clickhouse, Risingwave – both IaaS and DBaaS.
• Strong problem-solving skills and the ability to troubleshoot complex issues with the database infrastructure.
• Exceptional reporting skills, maintaining well-documented and current organizational data processing and transformation pipelines, ensuring their alignment with business objectives are clear and up-to-date.