Data Engineer

Requirements: 

This role is an individual contributor position, with a focus on hands-on data engineering tasks.

• Data Architecture Design: Design and implement scalable and efficient data architecture, including data models, data warehouses, and data lakes. Understand the business requirements and design appropriate data models, data pipelines, and data warehouses.
• Data Integration: Integrate data from various sources such as databases and APIs into a unified format for analysis. Develop ETL (Extract, Transform, Load) processes and real-time data pipelines.
• Data Modeling: Design and implement dimensional and data models to support data warehouses, and analytical and reporting needs.
• Data Pipeline Development: Develop and maintain robust ETL processes and data pipelines for ingesting, processing, and transforming large volumes of data from various sources. Ensure data quality, reliability, and consistency throughout the pipelines.
• Performance Optimisation: Optimise the performance of data processing, visualization, and storage systems, including database tuning, query and ETL processes optimisation, and infrastructure scaling, to ensure timely and efficient data access.
• Data Governance and Security: Establish and enforce data governance policies and procedures to ensure data integrity, privacy, and compliance with regulations and internal policies. Manage access controls, encryption, and auditing of data.
• Tool and Technology Selection: Evaluate and select appropriate tools and technologies for data storage, processing, and visualisation.
• Collaboration and Communication: Collaborate with cross-functional teams such as data scientists, analysts, and business stakeholders to understand their requirements and deliver data solutions that meet their needs. Communicate technical concepts effectively to non-technical audiences.
• Documentation and Knowledge Sharing: Document data pipelines, processes, and best practices to facilitate knowledge sharing and ensure the maintainability of data solutions. Promote a culture of documentation and knowledge sharing within the team.
• Continuous Learning and Improvement: Stay updated with the latest trends, advancements, technologies, and best practices in data engineering through continuous learning and self-improvement.