Data Engineer

We are looking fora Data Engineerfor ourprestigious client in theIT Services and IT Consultingindustry. This is a hybrid position based in Kuala Lumpur, Malaysia.

Key Responsibilities:

• Design, develop, and maintain scalable, automated, and reliable data pipelines using AWS Glue, ensuring efficient data ingestion, transformation, and loading (ETL) processes.
• Utilize AWS Kinesis for real-time data streaming and processing, enabling timely insights and actions based on streaming data sources.
• Manage and optimize AWS Redshift clusters for high-performance and cost-effective data warehousing solutions, ensuring data is easily accessible for analytics and business intelligence purposes.
• Work closely with stakeholders to understand data requirements and implement systems that support data collection, integration, and analysis across various sources and formats.
• Implement data retention policies, and ensure compliance with data security and privacy standards.
• Monitor and troubleshoot any data processing or performance issues, continually optimizing the data infrastructure for improved efficiency and reliability.
• Collaborate with data scientists, analysts, and other team members to support data modeling, query optimization, and the development of analytics tools and dashboards.
• Stay up-to-date with the latest AWS services and technologies, as well as industry best practices, to recommend and implement improvements to the data architecture.

Qualifications:

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
• 4+ years of data engineering experience, with a minimum of 2 years focused on AWS technologies including Glue, Redshift, and Kinesis.
• Hands-on experience building and optimizing data pipelines, architectures, and data sets using AWS.
• Strong analytic skills related to working with unstructured datasets and converting them into actionable insights.
• Proficiency in scripting languages (Python/Scala) for data processing and analytics.
• Experience in real-time data processing and analytics using AWS Kinesis.
• Solid understanding of data warehousing principles, and ETL processes.
• Knowledge of additional AWS services (e.g., S3, Lambda, and IAM) and data integration techniques is highly desirable.