Senior/Lead Databricks Platform Engineer

​​​​​​Required Expertise

Seeking an accomplished Sr./Lead AWS Cloud Engineer or Administrator with proven experience in solution architecture and Databricks platform administration. Experience with Azure is highly regarded.

Databricks

Extensive hands-on expertise with Databricks is essential.

Position Overview

We are looking for an experienced Enterprise-level Databricks Platform Engineer possessing a comprehensive understanding of both Databricks and leading cloud platforms.

Key Responsibilities

Platform Design & Implementation

• Architect, deploy, and oversee Databricks workspaces on major cloud providers (Azure, AWS).


• Develop scalable, secure, and reliable cloud-based data solutions.

Collaboration & Support

• Collaborate with multidisciplinary teams including DevOps, Data Science, and Business Intelligence to align platform capabilities with organizational objectives.


• Advise on best practices for Databricks implementation and data architecture.

Security & Compliance

• Apply robust security controls, including access management, encryption, and data masking.

Automation & Innovation

• Create Infrastructure as Code (IaC) scripts using tools such as Terraform.


• Lead initiatives to automate deployment, scaling, and monitoring of cloud-based data platforms.

Required Skills & Qualifications

Experience

• Minimum 4–5 years of experience in cloud platform engineering, solution architecture, or similar roles.


• At least 5 years of direct experience working with Databricks.

Technical Competencies

• Deep knowledge of Databricks configuration and administration.


• Proficiency with at least one major cloud platform (AWS or Azure).


• Familiarity with IaC tools (e.g., Terraform, CloudFormation).


• Strong Python automation and scripting skills.

Additional Skills

• Strong understanding of CI/CD pipelines and DevOps methodologies.


• Experience with monitoring and logging solutions for cloud platforms.


• Advanced troubleshooting and problem-solving abilities.

Preferred Qualifications

• Certifications related to Databricks, AWS, or Azure.


• Understanding of security frameworks such as SOC 2, GDPR, or HIPAA.