Resident Solution Architect Databricks

Company : Unison Group

Job Type : Full Time

Sydney, New South Wales

Job Description - Resident Solution Architect Databricks

Design and implement large-scale Azure and Databricks Data Lakehouse solutions.
Build and optimize ETL/ELT pipelines for batch and real-time streaming workloads using Azure Data Factory, Databricks, and Apache Spark.
Develop scalable data ingestion frameworks and integrate diverse structured and unstructured data sources.
Optimize data storage and query performance using Delta Lake, Parquet format, partitioning, and Spark performance tuning techniques.
Implement robust data governance, security, and access control using Unity Catalog, Azure Key Vault, and least-privilege principles.
Build and maintain data quality frameworks using Great Expectations or similar validation tools for batch and streaming data.
Automate pipeline deployment and CI/CD processes using Azure DevOps, Git, and configuration management tools.
Enable advanced analytics and ML workflows by preparing curated datasets and integrating with ML platforms like MLflow.
Develop real-time streaming solutions using Kafka, Azure Event Hubs, and Databricks Structured Streaming.
Create reusable PySpark libraries and frameworks for data curation, reconciliation, notifications, and Delta table automation.
Migrate legacy Hive or on-prem data sources to cloud platforms while ensuring security, compliance, and performance.
Monitor pipeline performance, troubleshoot issues, and implement alerting and logging for proactive management.
Collaborate with cross-functional teams and stakeholders to translate business requirements into scalable data solutions.
Establish best practices, standards, and documentation for enterprise-wide data engineering and analytics initiatives.
Mentor junior engineers and promote knowledge sharing within the data engineering team.

Original job Resident Solution Architect Databricks posted on GrabJobs ©. To flag any issues with this job please use the Report Job button on GrabJobs.

Share Job

Get your Resume Reviewed for Free

Similar Solution Architect Jobs in Australia

Get your Resume Reviewed for Free

Email address

Why are you reporting this job?

I think it’s a discriminatory or offensive

I think it’s fraudulent or a scam

I think it’s trying to sell something unrelated to the job / it’s asking for money

I think it contains incorrect or broken information

Other

All Job Ads are subject to GrabJobs’s Terms of Service. We allow users to flag postings that may be in violation of those terms. Job Ads may also be flagged by GrabJobs moderation team. However, no moderation system is perfect, and flagging a posting does not ensure that it will be removed.

Setup your job alert:

Frequency

By activating job alerts, I agree to GrabJobs Terms & Privacy Policy. I can unsubscribe to job alerts anytime. Skip

Resident Solution Architect Databricks

Job Description - Resident Solution Architect Databricks

Similar Solution Architect Jobs in Australia

Mobile Apps