Databricks Architect - Contract - Brisbane

Requirements

• Seeking a highly skilled Databricks Architect to lead the migration of enterprise data workloads from Greenplum to Databricks.
  


• This role requires deep expertise in distributed data systems, cloud architecture, and data engineering best practices.
  


• The architect will be responsible for designing scalable, secure, and high-performance data pipelines and lake house solutions on Databricks.
  

• Lead the end-to-end migration strategy from Greenplum to Databricks.
  


• Analyze existing Greenplum schemas, workloads, and performance metrics.
  


• Design target architecture using Databricks Lakehouse, Delta Lake, and Spark.
  


• Develop scalable ETL/ELT pipelines using PySpark, SQL, and Databricks Workflows.
  


• Ensure data quality, integrity, and consistency throughout the migration process.
  


• Collaborate with data engineers, DBAs, and business stakeholders to validate and optimize migrated data.
  


• Implement best practices for data governance, security, and compliance in Databricks.
  


• Provide technical leadership and mentorship to the migration team.
  


• Document architecture decisions, migration plans, and operational procedures.
  

• Bachelor’s or master’s degree in computer science, Information Systems, or related field.
  


• 8+ years of experience in data engineering or architecture roles.
  


• Proven experience with Greenplum and MPP systems.
  


• Experience with data migration tools and frameworks.
  


• Strong hands-on experience with Databricks, Apache Spark, and Delta Lake.
  


• Proficiency in SQL, Python, and PySpark.
  


• Experience with cloud platforms (preferably AWS).
  


• Knowledge of data modeling, performance tuning, and distributed computing.
  


• Certifications in Databricks or cloud platforms is preferred
  


• Excellent communication, leadership, and problem-solving skills.