Sr. Azure Data Architect Databricks

Overview :

Location: REMOTE Hybrid

Key Responsibilities

• Design and implement end-to-end data architecture / data ecosystem using Azure-native services and Databricks
  
• Build scalable, high-performance data platforms and robust data frameworks
  
• Partner with stakeholders to translate business requirements into reliable data solutions
  
• Implement data governance principles using Unity Catalog
  
• Create and implement data ingestion and data integration patterns using Medallion Architecture
  
• Apply best practices in data engineering
  
• Develop PySpark notebooks
  
• Create procedures for advanced data transformation
  
• Handle performance tuning in Databricks and Azure SQL DBs
  
• Architect end-to-end data solutions supporting:
  
  • Yield improvement
    
  • Process control
    
  • Predictive maintenance
    
  • Operational efficiency
    
• Ensure strict governance, security, scalability, and compliance across the platform
  

Required Experience

• 10+ years of overall experience and 5+ years of architecture experience in data architecture / data engineering roles
  
• Strong hands-on experience with major enterprise data platforms
  
• Extensive experience with Azure-native services including:
  
  • Azure Data Lake Storage (ADLS)
    
  • Azure Data Factory (ADF)
    
  • Azure Synapse Analytics
    
  • Azure SQL
    
• Strong hands-on experience with Azure Databricks
  

Required Databricks Expertise

Should have worked with the latest Databricks capabilities including:

• Unity Catalog:
  
  • Catalog and schema design
    
  • Permissions and security
    
  • Managed and external tables
    
  • Lineage and governance
    
• Delta Live Tables (DLT):
  
  • Reliable pipeline development
    
  • CDC implementation
    
  • Data transformations
    
  • Data quality controls
    
  • Scaling and performance tuning
    
• Other modern Databricks capabilities and best practices
  

Data Ingestion & ELT Experience

• Experience with data ingestion tools such as:
  
  • Fivetran
    
  • HVR or equivalent CDC / replication tools
    
• Experience with ELT tools such as:
  
  • DBT
    
• Familiarity with:
  
  • Real-time / streaming data
    
  • Batch processing
    
  • Sensor / OT / ET data flows
    

Requirements

Benefits