AWS Data Bricks_5+ years

We are looking for an experienced Databricks Engineer with strong expertise in Databricks, AWS, PySpark, and SQL. The ideal candidate should have hands-on experience in designing, developing, optimizing, and supporting large-scale data processing solutions on the Databricks platform. The role involves both development and production support responsibilities, requiring strong troubleshooting, performance tuning, and collaboration skills.

Key Responsibilities

• Design, develop, test, and deploy scalable data processing solutions using Databricks.


• Build and optimize ETL/ELT pipelines using PySpark and SQL.


• Work with AWS services, particularly S3, for data storage and processing.


• Develop high-performance Databricks solutions following industry best practices.


• Monitor and support production workloads, investigate issues, and implement fixes.


• Perform root cause analysis and provide timely resolution for production incidents.


• Collaborate with business stakeholders, architects, and development teams to understand requirements and deliver solutions.


• Recommend architecture improvements, code optimizations, and performance enhancements.


• Ensure data quality, reliability, and scalability across data platforms.


• Participate in code reviews, testing, deployment, and release activities.

Required Skills

• 7+ years of overall IT experience.


• 5+ years of hands-on experience with Databricks development.


• Strong experience with PySpark and SQL.


• Hands-on experience with AWS services, especially S3.


• Good understanding of Databricks architecture and optimization techniques.


• Experience in developing, debugging, and maintaining production data pipelines.


• Knowledge of data engineering best practices, performance tuning, and troubleshooting.


• Experience handling production support activities and incident management.


• Strong analytical and problem-solving skills.


• Excellent communication and stakeholder management skills.

Preferred Skills

• Databricks Certification.


• Experience in Retail domain projects.


• Strong understanding of ETL/Data Warehousing concepts.


• Exposure to DevOps practices and CI/CD processes.


• Experience working in both development and support environments.

Educational Qualification

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.

What We Are Looking For

• Self-driven professional capable of working independently.


• Strong ownership mindset with the ability to handle production-critical issues.


• Ability to identify opportunities for process improvements and platform optimization.


• Excellent collaboration and communication skills to work effectively with customers and cross-functional teams.


• 
  

  We are looking for an experienced Databricks Engineer with strong expertise in Databricks, AWS, PySpark, and SQL. The ideal candidate should have hands-on experience in designing, developing, optimizing, and supporting large-scale data processing solutions on the Databricks platform. The role involves both development and production support responsibilities, requiring strong troubleshooting, performance tuning, and collaboration skills.

  
  

  Key Responsibilities

  
  
  
  
  • Design, develop, test, and deploy scalable data processing solutions using Databricks.
  
  
  • Build and optimize ETL/ELT pipelines using PySpark and SQL.
  
  
  • Work with AWS services, particularly S3, for data storage and processing.
  
  
  • Develop high-performance Databricks solutions following industry best practices.
  
  
  • Monitor and support production workloads, investigate issues, and implement fixes.
  
  
  • Perform root cause analysis and provide timely resolution for production incidents.
  
  
  • Collaborate with business stakeholders, architects, and development teams to understand requirements and deliver solutions.
  
  
  • Recommend architecture improvements, code optimizations, and performance enhancements.
  
  
  • Ensure data quality, reliability, and scalability across data platforms.
  
  
  • Participate in code reviews, testing, deployment, and release activities.
  
  
  
  

  Required Skills

  
  
  
  
  • 7+ years of overall IT experience.
  
  
  • 5+ years of hands-on experience with Databricks development.
  
  
  • Strong experience with PySpark and SQL.
  
  
  • Hands-on experience with AWS services, especially S3.
  
  
  • Good understanding of Databricks architecture and optimization techniques.
  
  
  • Experience in developing, debugging, and maintaining production data pipelines.
  
  
  • Knowledge of data engineering best practices, performance tuning, and troubleshooting.
  
  
  • Experience handling production support activities and incident management.
  
  
  • Strong analytical and problem-solving skills.
  
  
  • Excellent communication and stakeholder management skills.
  
  
  
  

  Preferred Skills

  
  
  
  
  • Databricks Certification.
  
  
  • Experience in Retail domain projects.
  
  
  • Strong understanding of ETL/Data Warehousing concepts.
  
  
  • Exposure to DevOps practices and CI/CD processes.
  
  
  • Experience working in both development and support environments.
  
  
  
  

  Educational Qualification

  
  
  
  
  • Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
  
  
  
  

  What We Are Looking For

  
  
  
  
  • Self-driven professional capable of working independently.
  
  
  • Strong ownership mindset with the ability to handle production-critical issues.
  
  
  • Ability to identify opportunities for process improvements and platform optimization.
  
  
  • Excellent collaboration and communication skills to work effectively with customers and cross-functional teams.