Data Engineer_India_103054

What you'll be doing:

• Design, build, and deploy robust ETL and data management processes specifically for ingesting, transforming, and loading high-volume digital analytics data from Google Analytics 4 (GA4) into BigQuery/Snowflake.


• Develop and optimize BigQuery/Snowflake datasets, tables, and views to support various analytical needs, ensuring efficient querying and data integrity.


• Design, build, and deploy ETL job workflows with reliable error/exception handling and rollback frameworks. This includes designing and implementing data pipelines to feed data models for subsequent consumption.


• Monitoring and optimizing data processing and storage resources, with a focus on performance and cost efficiency.


• Troubleshooting and resolving data pipeline issues and performance bottlenecks, particularly those related to large-scale digital data processing.


• Write complex, customized SQL queries to manipulate data, generate automatic periodic reports, and support ad-hoc analytical requests.


• Build applications and scripts using Python to automate data processes, integrate systems, and enhance data quality.


• Develop strategies for data ingestion (from multiple sources) into data platforms, using various extract and load techniques through scripting and/or tooling, streaming, API consumption, and replication.


• Document data engineering processes, best practices, and technical specifications, especially for GA4 data models and BigQuery/Snowflake schemas.


• Work collaboratively with Business Partner Teams and Business Stakeholders during project scoping and feasibility phases as a Subject Matter Expert (SME) for concept investigation/viability and technical impact; identify and communicate technical risks and mitigations.


• Conform to agile development practices – Evolutionary design, refactoring, continuous integration/delivery, test-driven development.


• Provide production support for data load jobs, ensuring the continuous flow of critical digital data. Actively monitor and resolve user support issues, working closely with your functional squad and other squads.


• Maintain or upgrade existing data applications and pipelines.


• Clearly communicate technical terms to non-technical people and help them understand why change might be required to achieve a specific goal or to complete a project.


• Attend key design meetings and provide support.


• Perform research of viable technical and/or non-technical solutions.


• Other ad hoc duties as required.

What you’ll need:

• Education: Minimum of a Bachelor's degree in Computer Science, Engineering, Mathematics, or a related technical field preferred.


• Experience: 8+ years of relevant experience in data warehousing, data modeling, and building data integration pipelines, including ingestion, integration, and ETL.

Engineer's Core Skills:

• Proven experience in working with GA4 data, including understanding its data model, event-based tracking, and integration with BigQuery.


• 6+ years of hands-on experience with Google BigQuery or Snowflake including advanced SQL techniques (CTEs, window functions, aggregate functions), schema design, and performance optimization.


• 6+ years of hands-on experience with advanced SQL techniques (CTEs, window functions, aggregate functions).


• 5+ years of strong experience in Python programming (object-oriented/functional programming, Pandas, PySpark).


• 5+ years working with cloud platforms.


• Experience designing star schemas, analyzing data warehouses, and applying data warehouse methodologies, particularly in the context of digital analytics data.


• Essential experience in loading data into warehouse/ODS environments from diverse sources and formats. Design strategies for new data ingestion requests, including logical and physical data modeling.


• Hands-on experience with version control and CI/CD pipelines for data engineering workflows.


• Proven ability to perform unit testing (SQL scripts, ETL modules), integration testing, and performance/load/stress testing.

Tools & Technologies:

• Cloud Platforms: GCP (BigQuery, DataProc, DataFlow, CloudRun, Cloud Functions, Pub/Sub, etc.) or equivalent cloud platforms like AWS or Azure.


• Data Platforms: Snowflake.


• ETL/Integration Tools: AWS Glue, Fivetran.


• Orchestration/Transformation Tools: Alteryx or DBT (data build tool).


• Version Control: GitHub or similar repositories.


• BI Tools (Bonus): Looker or Power BI tools for data consumption.


• Other Web Analytics (Bonus): Experience with other web analytics platforms or marketing data sources.


• Infrastructure as Code (Bonus): Terraform.