AI focused DataBricks Developer (Digital Engineering)

Company Background

Our client is a leading life insurance company, part of a global financial services group. The company offers a wide range of products, including annuities and life insurance, delivered through a large nationwide network of agents.

Project Description

The project focuses on a production system managing clients and life insurance programs. The role involves both support and new feature development, with a strong emphasis on building and optimizing data pipelines and working within a modern Azure-based data platform.

Technologies

• Azure


• Databricks (Architecture, Unity Catalog, Delta, Lineage, Permissions, Databricks Genie, Notebooks)


• Agentic AI (Mosaic AI)


• Python


• SQL

What You'll Do

• Develop and maintain data pipelines within the Azure-based data platform;


• Work with the Databricks ecosystem (Delta Tables, Unity Catalog, notebooks) to process and manage data;


• Build and optimize data transformations using Python and SQL;


• Develop and support AI-driven data pipelines and automation using Databricks (Mosaic AI or similar tools);


• Support and enhance existing production data systems;


• Collaborate with a team of engineers on Databricks-based solutions;


• Ensure data quality, consistency, and reliability across data pipelines;

Job Requirements

• 3+ years of experience as a Data Engineer;


• Strong experience with Databricks ecosystem (Architecture, Unity Catalog, Delta tables, lineage, permissions, Databricks Genie, notebooks);


• Strong experience with Python;


• Solid proficiency in SQL;


• Experience working in Azure cloud environment;


• Experience with Databricks pipelines AI automation (Mosaic AI or AWS Agentcore);

Nice To Have

• Experience with Azure data integration tools (Synapse Pipelines, Azure Data Factory);


• Experience with Azure SQL pools;


• Understanding of performance tuning for data workloads;


• Experience with CI/CD, DevOps practices, Infrastructure as Code (IaC);

What Do We Offer

The global benefits package includes:

• Technical and non-technical training for professional and personal growth;


• Internal conferences and meetups to learn from industry experts;


• Support and mentorship from an experienced employee to help you professional grow and development;


• Health insurance;


• English courses;


• Sports activities to promote a healthy lifestyle;


• Flexible work options, including remote and hybrid opportunities;


• Referral program for bringing in new talent;


• Work anniversary program and additional vacation days.