Data Engineer with Talend, SQL and linux

Octal Philippines Inc. is seeking a skilled Data Engineer with strong expertise in Talend, SQL, and Linux to join our dynamic team. In this role, you will be responsible for designing, implementing, and maintaining data pipelines and workflows that facilitate data integration and transformation across various sources. You will work with cross-functional teams to ensure accurate and timely delivery of data to meet business needs.

Data Pipeline Development: Design and develop scalable data pipelines using Apache Spark, ensuring efficient data processing and transformation.

ETL Processes: Create and manage ETL (Extract, Transform, Load) workflows to integrate data from multiple sources into our data systems.

Data Integration: Utilize Talend for data integration tasks, including data cleansing, transformation, and loading.

Workflow Automation: Implement and manage data workflows using Apache Airflow to automate and optimize data processing tasks.

Programming: Leverage Python to write scripts for data manipulation, transformation, and analysis.

Database Management: Develop and optimize SQL queries for efficient data retrieval and analysis in relational databases.

Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand data needs and deliver effective solutions.

Data Quality: Ensure data accuracy, consistency, and quality through rigorous validation and monitoring processes.

Documentation: Create and maintain detailed documentation for data processes, pipelines, and integration workflows.

Required Skills and Qualifications:

Proficiency in Apache Spark for large-scale data processing and analytics.

Experience with ETL processes and tools, including Talend.

Familiarity with Apache Airflow for managing data workflows and automation.

Strong programming skills in Python for data engineering tasks.

Advanced SQL skills for querying and managing relational databases.