Data Engineer

JOB PURPOSE :

Responsible for the designing, developing, optimizing, and standardizing of batch and stream data pipelines, data platform and administration, data governance, data security, and modernization and migration of data platform to serve various use cases.

Principal Accountabilities

• Coordinate an agile data development team, taking responsibility for the organizing and planning required to fulfil customer data service needs.
• Liaise with development teams and product managers to ideate data software solutions.
• Implement, optimize, and manage databases.
• Build, optimize, and manage data platforms and Azure data services.
• Collaborate with relevant system/feature stakeholders to discuss, collect, interpret, and analyse data-related technical constraints and establish technical solutions.
• Develop data extraction and analysis products.
• Monitor Develop and manage databases.
• Write effective codes.
• Lead the Migration of the legacy data platform to Azure.
• Lead modernization of applications by working with the infrastructure team to migrate databases to Azure.
• Translating designs and wireframes into high-quality code.
• Perform peer code reviews.
• Monitor efficient and transferable technical documentation.
• Liaise with data scientists and analysts to improve and provide data as required.
• Lead the designing services/solutions, planning, and reviews of data solutions and products.
• Lead the refactoring of existing data pipelines for modernization and optimization.
• Collaborate with QA/QC team to ensure quality of testcases and testing process.
• Perform load testing and write unit test wherever needed.
• Monitor the use of Azure DevOps for version control of all projects
• Champion engagement with customers and all relevant stakeholders

Requisite Qualifications And Experience

• Bachelors degree in computer science/ Engineering or a related field
• Work experience 2- 3 years of cognate experience in a similar role in the technology industry is key.
• Advanced user of Azure data services, SQL Server Database, SQL Server Integration Services, SQL Server Analysis Services, Power BI, PYTHON, SQL
• Azure data ingestion tools such as Azure data factory, Azure synapse pipeline, Azure Event Hub, Azure IoT Hub etc.
• Excellent understanding of Azure networking
• Microsoft Certification in Azure Data Engineer
  

KEY COMPETENCE & BEHAVIOURAL EXPECTATIONS

• People management with strong leadership skills
• Excellent interpersonal, communication (written and verbal) and presentation skills
• Relationship management and negotiation skills
• Attention to detail and accuracy.
• Ability to handle multiple projects simultaneously and to tight timelines in a high-paced and dynamic environment.
• Excellent problem-solving and critical-thinking skills
• Dedication to meeting project deadlines.
• Effective coaching and mentoring skills