Data Engineer Associate - Finance and Supply Chain

Design, Develop and Deploy ETL/ELT Pipelines

• Work closely with Finance Business Users/SMEs, Business and Data Analysts to gather and translate the business requirements to clarify requirements and supports creation of technical specifications.
• Develop, Test and Deploy ETL/ELT pipelines for data ingestion, ingestion and loading using established data framework design and CI/CD practices.
• Implement automation for data workflows, job orchestration and dependency management either batch or real time data workflows.
• Participate in sprint ceremonies, release planning, development effort assessment and documentation creation and reviews.

Platform Operations and Support

• Provide Level 1/2 support for Finance DW pipelines which entails performing root cause analysis and code fix/enhancementacross data pipelines, storage layers and downstream integration/analytics requirements.
• Assist in monitoring production ETL/ELT jobs, execution of runbook and incidents monitoring.

Data Quality and Data Governance

• Supports the Data Quality Analyst to define and implement business rules, data validation, data integration rules and data quality/anomaly detection.
• Development of data validation, cleansing to ensure data accuracy and consistency.
• Ensure compliance with data governance standards and maintain metadata, data lineage and documentation.

REQUIREMENTS:

• Understanding of Microsoft Fabric components, specifically components of data lake and data warehouse (Lakehouse, Warehouse, Dataflows, Gen2, Pipeline, Notebooks).
• 1-2 years of overall experience in ETL/ETL development and including hands-on development and support of and data warehouse/data lake implementation.
• Familiarity of data warehouse concepts and data modelling, including dimensional modelling, medallion architecture (Bronze-Silver-Gold) and end to end (ETL/ELT) pipeline design.
• Basic knowledge of Python, Apache Spark, SQL and exposure to Cloud Native data integration tools.
• Demonstrated ability to collaborate effectively with cross-functional teams, including data scientists, analysts, and software engineers to deliver end-to-end data solutions and adaptability to agile environment.
• Exposure in implementing data quality, validation, and governance practices to ensure data integrity and compliance with organizational and regulatory and security standards.
• Familiarity with ERP system, budget and planning software and Finance Datawarehouse implementation is an advantage.
• Understanding of data governance and lineage. Experience with CI/CD practices using Git integration for Fabric.
• Working knowledge of PowerBI datasets and semantic modelling to enable BI and self-service.
• Experience with large data in a multinational organization. Strong willingness to learn and develop technical skills.
• Demonstrate capability to execute defined tasks and raises issues proactively.
• Has training, education or experience using AI