Data Engineer

Data Engineer/Developer

Role Summary
Design, build, and optimize Azure data pipelines and lakehouse solutions.
Deliver secure, reliable datasets with strong governance, automation, and
documentation. Collaborate across teams and contribute to standards in an Agile
setting.

Must -Have (Day 1)

• Experience:
  4–6 years in data engineering
  
• Core
  Platform: Databricks with Python, Spark, Pandas (notebooks and modular
  code)
  
• Orchestration:
  Azure Data Factory (pipelines, integration runtimes); ingest from diverse
  sources
  
• Lakehouse:
  Delta Lake fundamentals; Medallion architecture (bronze/silver/gold) in
  production
  
• Storage/SQL/Performance:
  Azure Data Lake Storage (ADLS); strong SQL; performance -aware design
  
• Data
  Patterns: ETL/ELT; data modeling (e.g., dimensional/star schema)
  
• DevOps
  & Security: CI/CD for data projects (Azure DevOps or GitHub
  Enterprise); familiarity with Azure Entra ID for SSO/RBAC; secure
  workspace/data access
  
• Quality
  & Observability: Data validation/testing, code reviews, and basic
  monitoring/alerting for jobs/pipelines
  
• Ways of
  Working: Agile/Scrum (Jira/Confluence); clear pipeline and data contract
  documentation
  
• Collaboration:
  Effective stakeholder engagement; support/mentor junior team members;
  clear communication
  
• Generative
  AI (Day 1):
  
• Prompt
  design for data tasks (ingestion, transformations, documentation) with
  clear objectives and constraints
  
• Use of
  Copilot/ChatGPT to scaffold notebooks/jobs, generate tests, and optimize
  SQL/Spark—validates outputs before merging
  

Nice -to -Have (Train within 60–90 days)

• Unity
  Catalog migration (Hive to Unity) and permissions/governance
  
• Databricks
  DevOps (cluster configuration, secret management, workspace automation)
  
• Azure
  Functions (C# or Python) for orchestration/integration
  
• Synapse
  dedicated SQL pools or dbt; Delta Live Tables
  
• Financial
  services domain exposure
  

Shared Expectations

• Work
  independently with minimal supervision while contributing to team outcomes
  
• Commitment
  to secure practices and production -grade reliability
  
• Continuous
  improvement mindset and willingness to learn new tools/technologies
  
• Willingness
  to work within regulated environment controls and policies
  
• Use
  Generative AI responsibly to improve velocity and quality (simple,
  structured prompts; guardrails; validate AI -assisted outputs before
  adoption)