Lead Engineer -Data Platform Administrator - Databricks

The Data Platform Administratoris responsible for leading software design, service architecture, and software development/solution engineering. You will manage junior engineers, lead design and code reviews within the service, and investigate emerging technologies to drive continuous improvement of the service:

Key Responsibilities:

• Administer, configure, and maintain the Databricks environment to ensure optimal performance and reliability.
• Collaborate with data engineering teams to design and implement effective data architectures, models, and pipelines within the Databricks platform.
• Monitor system performance, troubleshoot issues, and implement enhancements to improve reliability and efficiency.
• Develop and enforce best practices and security measures to protect data integrity and privacy.
• Utilize Databricks to facilitate efficient data integration, processing, and analytics across various sources.
• Manage user access, control permissions, and ensure compliance with internal and regulatory data governance standards.
• Optimize resource allocation and cost management within the Databricks environment.
• Provide technical leadership and support for data analytics and machine learning projects using Databricks.
• Train team members and stakeholders on the capabilities and utilization of Databricks to promote data-driven decision-making.
• Document technical procedures, configurations, and guidelines for database management and maintenance.
• Bachelors degree in Computer Science, Information Systems, or a related field.
• Minimum 3 years of experience in managing and configuring data platforms, with specific expertise in Databricks.
• Strong knowledge of Apache Spark, Delta Lake, and related big data technologies.
• Proficient in programming languages such as Python, Scala, or SQL.
• Experience with cloud services (AWS, Azure, or Google Cloud) and their integration with Databricks.
• Proven ability to work collaboratively in a team environment and communicate with various technical and non-technical stakeholders.
• Certifications in Databricks or related technologies are preferred.