Data bricks Lead Data Engineer - R01565550

Job Description – Data Engineer (Databricks)

Company: Brillio Technologies
Role: Data Engineer – Databricks
Experience: 5–10 Years
Location: India (Hybrid/3 days)
Employment Type: Full-time

 About Brillio

Brillio is a leading digital transformation partner delivering innovative solutions across Data, Cloud, AI, and Digital Engineering. We help global enterprises unlock value through cutting-edge technology and data-driven insights.

Role Overview

We are looking for a hands-on Data Engineer with strong Databricks expertise to design, build, and scale modern data platforms. The ideal candidate will have experience in PySpark, Delta Lake, and cloud-native data engineering and will work closely with analytics, product, and business teams to deliver high-performance data pipelines.

 Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines using Databricks and PySpark


• Build and optimize data lakehouse architectures (Bronze–Silver–Gold layers)


• Develop data ingestion frameworks from multiple sources (batch & real-time)


• Implement Delta Lake features including schema evolution, ACID transactions, and optimization


• Create and manage Databricks notebooks, workflows, and jobs for production pipelines


• Ensure data quality, governance, and lineage across pipelines


• Optimize Spark jobs for performance (partitioning, caching, tuning)


• Collaborate with cross-functional teams (Data Scientists, Analysts, Architects)


• Support production deployments, monitoring, and troubleshooting of data pipelines


• Contribute to migration of legacy data platforms to modern cloud-based architecture

Technical Skills (Must-Have)

• Strong experience in:




• Databricks (Workflows, Notebooks, Jobs)


• PySpark / Spark SQL


• SQL & Data Warehousing concepts




• Hands-on experience with:




• Delta Lake / Lakehouse architecture


• Data pipeline development & orchestration




• Good understanding of:




• Data modeling, ETL/ELT processes, SCD concepts 

Cloud & Tools

• Experience in at least one:




• AWS (S3, Glue, EMR) / Azure (ADF, ADLS, Synapse)




• Workflow tools:




• Airflow / Databricks Workflows




• Version control:




• Git / CI-CD pipelines

 Good to Have

• Experience with:




• Streaming frameworks (Kafka / Spark Streaming)


• DBT / Snowflake / Redshift / Fabric




• Exposure to:




• Unity Catalog, Autoloader, Delta Live Tables 

Experience Required

• 5–10 years of experience in Data Engineering


• Minimum 2–4 years of hands-on Databricks experience


• Proven experience building enterprise-scale data platforms and pipelines

Soft Skills

• Strong problem-solving and analytical skills


• Ability to work in a fast-paced, agile environment


• Good communication and stakeholder management

Why Join Brillio

• Work on cutting-edge data & AI programs


• Opportunity to build large-scale data platforms


• Collaborative and innovation-driven culture