Data Platform Engineer

Job: Data Platform Engineer (Snowflake and Databricks)

Location: North Dublin

Working model: Hybrid (1-2 days on site)

Rate: 450-475 per day

Type: Contract

Duration: 12 months+

We are working with a leader of industry. Working alongside a talented Engineering team, we are looking to add a talented Data Platform Engineer to join our team in Dublin.

This Job offers the chance to shape and support the teams data architecture, working on cutting-edge cloud technologies and driving the success of our data-driven projects. You should have a strong background in Databricks, Snowflake, and AWS. You should be proficient in MLOps to support seamless deployment and scaling of machine learning models. Youll play a critical role in our mission to enhance data accessibility, streamline data sourcing pipelines and optimize performance for large-scale data solutions.

Key responsibilities & duties include:

• Architect and Implement Cloud-Native Data Solutions: Design and develop scalable data platforms, focusing on a cloud-native approach, data mesh architectures, and seamless integration across multiple data sources
• MLOps Pipeline Development: Build and maintain MLOps pipelines using tools like MLflow, ensuring efficient and reliable deployment of machine learning models to production environments
• Data Governance and Quality Management: Create and enforce data governance standards, ensuring robust data quality and compliance through tools such as Databricks Unity Catalog
• Data Integration & Migration: Lead migration projects from legacy data platforms to modern cloud solutions, optimizing cost and operational efficiency
• Performance Tuning and Optimization: Leverage tools such as Snowflake and Delta Lake to improve data accessibility, reliability, and performance, delivering high-quality data products that adhere to best practices

Key Projects/Deliverables:

• Data Mesh Architecture: Design and deployment of data mesh frameworks to streamline data integration and scalability across business domains
• MLOps Pipelines: Prototype and operationalize MLOps pipelines to enhance the efficiency of machine learning workflows
• Data Migration & Cost Optimisation: Migrate large-scale datasets to Azure and AWS platforms, with a focus on business-critical data sources and significant cost reductions
• Data Governance Applications: Develop applications to enforce data governance, data quality, and enterprise standards, supporting a robust production environment

Required Experience:

• Experience in Data Platform Engineering: Proven track record in architecting and delivering large-scale cloud-native data solutions
• Proficiency in Databricks and Snowflake: Strong skills in data warehousing and lakehouse technologies with hands-on experience in Databricks, Spark, Pyspark and Delta Lake
• MLOps Expertise: Experience with MLOps practices, ideally with MLflow for model management and deployment
• Cloud Platforms: Knowledge of AWS, with additional experience in Azure beneficial for multi-cloud environments
• Programming Languages: Strong coding skills in Python, SQL and Scala
• Tooling Knowledge: Experience with version control (GitHub), CI/CD pipelines (Azure DevOps, GitHub Actions), data orchestration tools (Airflow, Jenkins) and dashboarding tools (Tableau, Alteryx)

• Some familiarity with data governance tools and best practices would be ideal.

This is an exciting opportunity to work in a secure contract with a leading business undertaking a major digital transformation. You will play a critical part of the technical advancement and work along side a skilled and friendly group of Engineers. If you would be interested, please submit your CV to the link provided for immediate consideration.