Data Engineer (Teradata SQL, Informatica, Apache Spark )

Develop and deliver scalable ETL/ELT solutions across Teradata and Hadoop platforms by building robust data pipelines, ingestion frameworks, and automation capabilities. Ensure efficient data processing, performance optimization, and reliable integration of enterprise data sources, while supporting data warehouse, CDC, and MDM initiatives.

Responsibilities

• Create EDW ETL/ELT codes using Teradata SQL, Informatica, Apache Spark, QueryGrid, Trino to perform various transformations and load into Teradata based data warehouse or datamarts
• Performance tuning of highly complex applications to reduce resource usage
• Create framework using GCFR, shell scripts, BTEQ and shell scripts to automate end to end. Also to integrate with control
• Build in-house CDC framework using Java to process OLR logs from SAP(Oracle) and ingestion the data into Teradata. Create framework around this CDC ingestion to make it configurable
• Install and build Master Data Management (MDM) application for various user uploads along with data validations and approval workflow.

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Engineering, Information Technology, or a related field.
• Minimum of 7+ years of professional experience
• Strong hands‑on experience with the ETL(Informatica Power Center) and Teradata based ETL development, similarly Hadoop ecosystem (Hive, Impala, Spark, Kafka, Iceberg, Ranger, Atlas, Nifi, Flink etc.,), and data pipeline orchestration.
• Create EDW ETL/ELT codes using Teradata SQL, Informatica, apache Spark, QueryGrid, Trino to perform various transformation and load into Teradata based data warehouse or datamarts
• Performance tuning of highly complex applications to reduce resource usage
• Create framework using GCFR, shell scripts, BTEQ and shell scripts to automate end to end. Also to integrate with control
• Build in-house CDC framework using Java to process OLR logs from SAP(Oracle) and ingestion the data into Teradata. Create framework around this CDC ingestion to make it configurable

Install and build Master Data Management(MDM) application for various user uploads along with data validations and approval workflow.

Key Skills:


• ETL/ELT Development using Informatica Power Center and Teradata
• Data Platform Expertise
• Data Pipeline & Orchestration using enterprise scheduling tools
• Performance Optimization
• Data Integration & CDC
• MDM – Reference data management
• ETL Framework & Automation Development