Design, build, and operate end-to-end data pipelines (ingestion → transformation → serving) on Databricks and/or Snowflake, following medallion/layered architecture patterns
Configure and administer workspaces/accounts - compute policies, resource sizing, environment setup, and access hierarchy - per the SDP reference architecture
Implement data governance controls: catalogue and schema design, RBAC, row/column-level security, data masking, and lineage tracking
Set up CI/CD and infrastructure-as-code for pipeline deployment and environment promotion (dev → test → prod)
Configure monitoring, telemetry, and audit logging to meet SDP's central observability and security posture requirements
Support UAT, integration testing, and parallel-run validation during migration and go-live
Produce handover documentation (runbooks, access lists, escalation procedures) for agency operations teams
Required Technical Skills - Databricks
Unity Catalog - catalogue/schema design, access control, and data lineage
Lakeflow / Delta Live Tables for pipeline orchestration; Delta Lake table format
Databricks SQL and cluster/workspace administration (compute policies, pools, cost management)
Databricks Asset Bundles (DABs) and Databricks Repos for CI/CD
PySpark / Spark SQL for large-scale data transformation
Working knowledge of Databricks system tables (audit logs, billing/usage, query history) for observability
Minimum 5 years of hands-on experience in Data Engineering, Data Platform Engineering, or related disciplines.
Minimum 3 years of hands-on experience with Databricks involving data pipeline development, platform administration, governance, and optimization.
Required Technical Skills - Common
Strong SQL and Python (PySpark or general-purpose) for data engineering
Data modeling - dimensional design, star/snowflake schemas, semantic layers
CI/CD pipelines (e.g., Azure DevOps, GitHub Actions, GitLab CI) for data engineering workflows
Infrastructure-as-code (Terraform preferred) for provisioning cloud data platform resources
Hands-on experience on at least one hyperscaler - AWS, Azure, or Google Cloud
Understanding of data security and compliance frameworks
Preferred Qualifications
Databricks Certified Data Engineer Associate/Professional
SnowPro Core, or SnowPro Advanced: Data Engineer
AWS certification (e.g. AWS Certified Data Engineer - Associate, Solutions Architect) and hands-on experience with AWS data services.
Prior experience delivering on a regulated-sector data platform, or exposure to compliance frameworks such as IM8, is an add-on
Experience working as part of a System Integrator (SI) delivery team alongside a platform Principal is an add-on
All Job Ads are subject to GrabJobs’s Terms of Service. We allow users to flag postings that may be in violation of those terms. Job Ads may also be flagged by GrabJobs moderation team. However, no moderation system is perfect, and flagging a posting does not ensure that it will be removed.
Be the first to receive the latest Others Full-Time Jobs in Singapore.
Setup your job alert:
By activating job alerts, I agree to GrabJobs Terms & Privacy Policy. I can unsubscribe to job alerts anytime.
Skip
GrabJobs is the no1 job portal in Singapore, connecting you to thousands of jobs fast!
Find the best jobs in Singapore, apply in 1 click and get a job today!