Databricks - Data Engineer

Requirements

Security Clearance:    None/Secret/Top Secret

Location :   Remote

Certifications:

      - Databricks Certified Data Engineer Associate certification 

Education: 

• Bachelor's degree in a relevant field or equivalent combination of education and experience 
  
• 7+ years of experience in the data engineering field 
  
• 3+ years of experience in a data analytics environment, preferably in DoD or the intelligence community 
  
• Master's degree in a relevant field (may substitute for 3 years of general experience) 
  

Required Skills/Experience:

• Previous experience in a data engineering role or similar 
  
• Experience with data processing and pipeline management 
  
• Familiarity with industry -standard data engineering platforms and tools 
  
• Strong data manipulation and processing skills 
  
• Proficiency in data engineering tools such as Databricks, Apache Spark, Delta Lake, MLflow, and SQL 
  
• Understanding of the Databricks Lakehouse Platform, its workspace, architecture, and capabilities 
  
• Ability to perform multi -hop architecture ETL tasks using Apache Spark SQL and Python 
  
• Knowledge of incremental data processing in batch and streaming mode 
  
• Familiarity with open -source tools, cloud computing, machine learning, and data visualization 
  
• Strong interpersonal skills and a collaborative work style 
  
• Understanding of security and governance best practices
  
• Strong problem -solving and analytical skills 
  
• Excellent written and verbal communication abilities
  
  

Major Duties/Tasks: 

• Build and manage data pipelines for data engineering applications 
  
• Process and manipulate large datasets efficiently and securely 
  
• Work with data engineering platforms and tools such as Databricks, Apache Spark, Delta Lake, MLflow, and SQL 
  
• Maintain best practices around security and governance 
  
• Model data management solutions 
  
• Manage, test, and deploy code 
  
• Provide expertise on data concepts to the advanced analytics group 
  
• ​Install continuous pipelines of filtered information for data analysts and scientists to access relevant datasets 
  
• Collaborate with cross -functional teams to ensure effective data utilization
  

Benefits

• 401K Plan
  
  
• Vacation and Paid Time Off
  (PTO)
  
  
• Health, Dental & Vision
  Insurance
  
  
• Life & Supplemental Life
  Insurance
  
  
• Disability & Accidental
  Death & Dismemberment
  


• Mental Health Care
  
  
• Health Saving Account (HSA)