Lead Technical Consultant - Databricks

Lead Technical Consultant - Databricks

Argano is seeking a Lead Technical Consultant - Databricks to lead the technical design and delivery of mid- to large-scale data engineering and analytics solutions using the Databricks Lakehouse Platform. This is a hands-on, client-facing role that combines solution design, development, workstream leadership, quality oversight, and team mentorship. The Lead Technical Consultant translates business and technical requirements into scalable solutions, guides implementation teams, and independently delivers complex project work while partnering with project managers, architects, engineers, analysts, and client stakeholders.

RESPONSIBILITIES

• Lead the technical delivery of Databricks workstreams across requirements, design, development, testing, deployment, and transition to support.


• Gather and analyze business and technical requirements and translate them into practical solution designs, implementation plans, and technical specifications.


• Design and build scalable data pipelines and lakehouse solutions using Databricks, Apache Spark/PySpark, Python, SQL, and Delta Lake.


• Lead the deployment and integration of Databricks with cloud data platforms, source systems, data warehouses, analytics tools, and downstream applications.


• Configure and optimize Databricks jobs, workflows, clusters, notebooks, compute resources, and data pipelines for performance, reliability, scalability, and cost efficiency.


• Implement data governance, security, access controls, lineage, and compliance practices using Unity Catalog and related cloud services.


• Establish development standards and lead design reviews, code reviews, unit testing, documentation, version control, CI/CD, and troubleshooting within the assigned workstream.


• Lead estimation activities, identify technical dependencies and risks, and help define realistic development timelines and delivery goals.


• Guide and mentor consultants and engineers, delegate technical tasks, and ensure the team delivers consistent, high-quality work.


• Collaborate with client and cross-functional stakeholders, produce technical documentation, and facilitate solution walkthroughs, issue resolution, knowledge transfer, and enablement.


• Evaluate relevant Databricks capabilities and emerging technologies and recommend practical improvements that strengthen solution quality and delivery.

MINIMUM AND/OR PREFERRED QUALIFICATIONS

Education

• Bachelor's degree in Computer Science, Software Engineering, Information Technology, Data Engineering, or a related field; equivalent relevant experience may be considered.

Experience

• 5-10 years of experience in technical consulting, data engineering, analytics engineering, software development, or related technology roles.


• At least 3 years of hands-on experience designing, developing, and delivering solutions on the Databricks platform.


• Demonstrated experience leading a technical workstream or small delivery team on mid- to large-scale client implementations.


• Strong hands-on experience with Python, SQL, Apache Spark/PySpark, ETL/ELT frameworks, and data pipeline development.


• Experience with key Databricks capabilities such as Delta Lake, Unity Catalog, Workflows/Jobs, notebooks, compute or cluster management, and platform administration concepts.


• Experience integrating Databricks with at least one major cloud platform: Microsoft Azure, Amazon Web Services (AWS), or Google Cloud Platform (GCP).


• Strong understanding of lakehouse, data lake, data warehouse, batch and streaming processing, data modeling, and modern analytics concepts.


• Experience with Git-based development, DevOps, automated testing, CI/CD, Agile delivery, and managing multiple priorities or workstreams.


• Client-facing consulting experience, including requirements discussions, technical workshops, solution presentations, estimation, and delivery communication.

Preferred Certifications and Experience

• Databricks Data Engineer Associate, Databricks Data Engineer Professional, or another relevant Databricks certification.


• Experience with MLflow, machine learning workflows, real-time or streaming solutions, or Databricks SQL.


• Experience supporting technical discovery, solution estimates, proposals, or statements of work in partnership with architects and sales teams.

SKILL REQUIREMENTS

• Technical Leadership: Ability to set direction for a workstream, guide team members, and remain hands-on throughout delivery.


• Problem-Solving and Judgment: Strong analytical and troubleshooting skills with the ability to prioritize, identify risks, and make sound technical decisions.


• Solution Design and Quality: Ability to design scalable, maintainable, secure, and cost-conscious solutions while reinforcing testing, documentation, governance, and engineering standards.


• Communication and Client Engagement: Excellent verbal and written communication skills and the ability to build trust and explain technical concepts to varied audiences.


• Mentorship and Collaboration: Ability to coach team members and work effectively with project managers, architects, engineers, analysts, and business stakeholders.


• Accountability: Ability to work independently and take ownership of assigned technical outcomes and delivery commitments.