We are seeking a skilled Data Engineer with expertise in Google Cloud Platform (GCP) to design, build, and maintain scalable data pipelines and data warehousing solutions. This role is critical in enabling data -driven decision -making across the organization by ensuring reliable, efficient, and secure data processing across batch and streaming workloads. The ideal candidate will leverage tools such as BigQuery, Cloud Composer/Airflow, Dataflow, Dataproc, and Apache Beam to deliver end -to -end data solutions, while applying strong data modeling principles including SCD, partitioning, and clustering. Experience with event -driven architectures using Pub/Sub and Kafka integration or migration is highly valued. Additionally, familiarity with BigQuery ML and Vertex AI enables advanced analytics and machine learning capabilities. This position is based in Hyderabad and requires 6 -11 years of hands -on experience in cloud -based data engineering.
Location: Hyderabad
Responsibilities:
Design, develop, and maintain end -to -end batch and streaming data pipelines using BigQuery, Cloud Composer/Airflow, Dataflow, and Dataproc.
Implement robust data warehousing solutions using BigQuery with advanced techniques such as SCD, partitioning, and clustering.
Build and manage event -driven data workflows using Pub/Sub and integrate with Kafka for data ingestion or migration scenarios.
Write efficient, scalable SQL and Python code to process and transform large -scale datasets.
Leverage Apache Beam for pipeline development and ensure consistency across Dataflow -based implementations.
Collaborate with data scientists and analysts to support advanced analytics and ML initiatives using BigQuery ML and Vertex AI.
Ensure data reliability, performance, security, and observability across all data systems.
Participate in architecture discussions, code reviews, and continuous improvement of data infrastructure.
Requirements
6 -11 years of professional experience in data engineering with a strong focus on GCP technologies.
Expertise in BigQuery, Cloud Composer/Airflow, Dataflow, Dataproc, and Apache Beam (explicit or demonstrated via Dataflow pipelines).
Proven experience building end -to -end batch and streaming data pipelines.
Deep understanding of data warehouse concepts, including SCD, partitioning, and clustering.
Strong proficiency in Python and SQL for data transformation and pipeline development.
Hands -on experience with Pub/Sub for event -driven data processing.
Experience with Kafka, including integration or migration use cases.
Basic exposure to BigQuery ML and Vertex AI for predictive analytics and machine learning workflows.
Familiarity with cloud -native best practices, CI/CD, monitoring, and infrastructure as code.
All Job Ads are subject to GrabJobs’s Terms of Service. We allow users to flag postings that may be in violation of those terms. Job Ads may also be flagged by GrabJobs moderation team. However, no moderation system is perfect, and flagging a posting does not ensure that it will be removed.
Be the first to receive the latest Others Full-Time Jobs in India.
Setup your job alert:
By activating job alerts, I agree to GrabJobs Terms & Privacy Policy. I can unsubscribe to job alerts anytime.
Skip