Lead Data Engineer

Engineering - Software (Information & Communication Technology)

• Design and maintain a scalable and secure Azure data warehouse.
• Build data pipelines: Create and manage data integration processes, including data extraction, transformation, and loading (ETL) from various sources into the data warehouse.
• Develop and implement data management strategies to ensure data quality, consistency, and accuracy.
• Collaborate with cross-functional teams, including to identify data requirements and develop data models. 
• Develop, adhere, and enforce data governance policies and procedures to ensure compliance.
• Monitor data quality and develop response mechanisms to address problems. 
• Collaborate with IT teams to ensure the availability, reliability, and performance of data systems and infrastructure.
• Develop documentation related to data management processes, procedures, and data dictionaries.
• Stay up to date with industry trends and advancements in data management technologies and techniques.
• Maintain the current state legacy Microsoft environment and processing, which includes troubleshooting production issues and implementing tactical enhancements where needed.
• As needed, collaborate with external contractors to enhance data solutions. Coordinate their work, provide guidance, and ensure alignment with project goals.

Qualifications:

Education/Experience

• Education and Experience :
• Technical Skills :
• Data Privacy and Security :
• Problem-Solving and Analytical Skills :
• Effective Communication :
• Positive Attitude and Adaptability :
• Bachelor’s or Master’s degree in Computer Science, Information Systems, or a related field.
• Proven experience working as a Data Engineer or in a similar role, preferably within the insurance or financial sector.
• Strong understanding of  data management principles , including data modeling, schema design, and data lineage.
• Knowledge of  data quality assurance techniques and best practices.
• Familiarity with  data governance frameworks and policies.
• Proficiency in  data warehousing concepts and hands-on experience with Azure services (e.g., Azure Data Factory, Azure Synapse Analytics, Azure SQL Database).
• Ability to design and implement  ETL processes using Azure tools.
• Proficiency in  SQL for data manipulation and querying.
• Experience with  Python for data transformation and scripting.
• Familiarity with  ETL frameworks (e.g., Apache Spark, Talend).
• Solid understanding of regulatory requirements such as  GDPR and  HIPAA .
• Ability to implement data privacy controls and ensure secure data handling.
• Excellent ability to  analyze complex data sets , troubleshoot issues, and optimize data workflows.
• Creative problem-solving mindset to address data-related challenges.
• Strong  interpersonal skills to collaborate with cross-functional teams.
• Ability to convey complex technical ideas clearly to both technical and non-technical stakeholders.
• A  can-do mindset and willingness to tackle challenges head-on.
• Adaptability to evolving technologies and business needs.
• Data Processing and Analytics :
• Data Governance and Security :
• Monitoring and Troubleshooting :
• Scripting and Automation :
• Effective Communication and Collaboration :
• Continuous Learning and Adaptability :
• Support and maintenance for legacy technologies
• Azure Data Factory : Ability to design and implement data integration workflows using Azure Data Factory.
• Azure Data Lake Storage : Familiarity with managing data in Azure Data Lake Storage.
• Azure SQL Data Warehouse : Expertise in designing and optimizing data warehousing solutions using Azure SQL Data Warehouse.
• Azure Data Lake Analytics : Understanding of data processing using Azure Data Lake Analytics.
• Big Data Technologies : Knowledge of big data frameworks (e.g., Apache Spark, Hadoop) within Azure.
• Data Governance Principles : Awareness of data governance best practices within Azure.
• Access Control : Understanding of role-based access control (RBAC) and permissions management.
• Data Privacy Compliance : Knowledge of compliance requirements (e.g., GDPR, HIPAA) related to data privacy and security.
• Azure Data Factory Monitoring : Skills in monitoring, optimizing, and troubleshooting data pipelines within Azure Data Factory.
• Diagnostic Logging : Familiarity with diagnostic logs and metrics for Azure services.
• PowerShell or Python : Proficiency in scripting languages (PowerShell or Python) for automating tasks within Azure.
• Cross-Functional Teams : Strong communication skills to collaborate effectively with data scientists, analysts, and business stakeholders.
• Technical Documentation : Ability to document data solutions clearly.
• Stay Updated : Proactively keep up with the latest Azure data services, features, and advancements.
• Adaptability : Be ready to embrace new technologies and adapt to evolving business needs.
• Microsoft SQL Server : Knowledge of SQL Server for data warehousing and relational databases.
• T-SQL : Proficiency in Transact-SQL (T-SQL) for querying and managing data.
• Understanding of operating systems such as:
• Microsoft Windows
• Linux
• UNIX

What's your expected monthly basic salary?

How many years' experience do you have as a Data Engineer?

Report this job advert

Don’t provide your bank or credit card details when applying for jobs.