Data Engineer

Tech in Asia is building up our internal analytics products to deliver better data and insights to our users and editorial team.

We are looking for an experienced Data Engineer to join our growing team, who will be responsible for expanding and optimizing our data and data pipeline architecture, as well as optimizing data flow and collection for cross functional teams. The Data Engineer will support our software developers, database architects, data analysts and data scientists on data initiatives and will ensure reliability and quality of data. They must be self-directed and will be excited by the prospect of optimizing or even re-designing our company’s data architecture to support our next generation of products and data initiatives.

Work with data and analytics experts to strive for greater functionality in our data systems

Key Responsibilities:

• Data Architecture : Design and implement scalable data solutions, including databases, storage systems, and ETL/ELT processes, with a focus on long-term maintainability.
• Data Reliability & Monitoring: Establish processes for data validation, error handling, and disaster recovery. Develop dashboards and tools to monitor data health and proactively identify performance issues.
• Process Improvement: Identify areas to automate manual tasks, enhance data delivery efficiency, and streamline workflows.
• DevOps Integration: Collaborate with development teams to deploy and manage data solutions using containerization (Docker, Kubernetes) and cloud technologies (ideally AWS).
• Stakeholder Collaboration: Work closely with product, marketing, and leadership teams to understand their data needs and provide technical support.

Requirements:

• 2+ years of experience in a Data Engineer role or a similar data-focused position. Bachelor’s degree in Computer Science, Statistics, Informatics, Information Systems, or a related quantitative field preferred.
• Experience with object-oriented/object function scripting languages such as : Python, Java, C++, Scala, etc.
• Advanced working SQL knowledge and experience working with relational databases and familiarity with a variety of databases (eg. Postgres, MySQL, Redshift, MongoDB, Vector databases)
• Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.
• A successful history of manipulating, processing and extracting value from large disconnected datasets.
• Experience with data pipeline and workflow management tools: Airflow, Azkaban, Luigi, etc.

Highly desirable: 

• Experience with software development processes such as CI/CD pipelines and container management technologies like Docker and Kubernetes
• Experience with cloud computing platforms (AWS, GCP, Azure)Familiarity with web scraping principles and tools for data acquisition.