Big Data Administrator - Part-Time

Summary of Position

 He/She will be a technical L3 resource for all Big Data services and will provide support for all production support activities within the Big Data team in Singapore. He/She will work with L3/Service Manager to gain control over the scope of technical activities, develop best practices and gain knowledge overall aspects of support.

Responsibilities:

As a L3 resource of his /her team, he /she:

• Takes up technical tasks and also manages delegation for technical issues within the team, animates the team to encourage collaboration and sharing of best practices,
• supports new technologies and leverages them to provide consistency of service across streams,
• proposes service improvements for all Big Data services supported throughout the organization,
• documents, reviews, maintains and shares relevant technical information within the team
• provides technical knowledge, supports services both proactively and reactively to maintain the availability and reliability of system infrastructure in accordance to the SLA,
• Actively engages during any high severity issue and drives for issue resolution
• reviews technology changes to identify potential risks,

As an experienced professional in Big Data Services, he/she:

• supports his/her team during diagnosis when technical issues rise in his/her scope of expertise,
• is aware of the global IT structure so that he/she anticipates interrelationships within the organization,
• engages with technical peer, Development team, Service managers, Architect and project teams on technology roadmap and projects,

• facilitates transformation projects and suggest future directions for new areas of improvement and change,
• guarantees the production readiness and license to operate of new projects and solutions

• is available and able to drive technically, any complex or high severity incidents that occur within the scope of their role.

Candidate profile

Mandatory track record

• Excellent working knowledge of Hadoop ecosystem (Hadoop, Hive, Pig, Oozie, Hbase, Flume, sqoop) using both automated tool sets as well as manual processes.
• Support and maintain Hadoop (HDP) clusters for batch processing, analytics, and regulatory reporting.
• Perform Hadoop cluster lifecycle management: provisioning, scaling, patching, and decommissioning nodes.
• Ensure 24x7 availability and resilience of production systems supporting payment flows.
• Manage and optimize Apache Kafka for high-throughput, real-time payment event streaming.
• Ensure data consistency and fault tolerance across streaming pipelines.
• Support Apache NiFi for ingestion pipelines from upstream payment systems and external partners.
• Work with AWS EMR for scalable processing of transaction data and reconciliation workloads.
• Administer HDFS, ensuring optimal replication, storage utilization, and fault tolerance.
• Monitor and tune MapReduce and YARN workloads to handle large-scale transaction data efficiently.
• Ensure proper configuration and validation of jobs handling payment clearing, settlement, and reporting.
• Manage OpenSearch / Elasticsearch clusters for transaction search, audit trails, and operational dashboards.
• Excellent working knowledge of Elastic Kibana and making pipelines to capture and search relevant data.
• Optimize indexing and query performance for near real-time analytics and monitoring.
• Implement Kerberos-based authentication and secure access controls across the Hadoop ecosystem.
• Manage user provisioning (Linux + Hadoop stack) ensuring least-privilege access.
• Ensure compliance with banking regulations, audit requirements, and data governance policies.
• Monitor cluster security, encryption, and network connectivity.
• Conduct capacity planning aligned with transaction growth and peak payment volumes.
• Optimize systems for low latency and high throughput, critical for digital payments.
• Identify bottlenecks and implement performance tuning strategies across platforms.
• Ensure high availability through failover mechanisms, DR strategies, and proactive monitoring.
• Develop and maintain runbooks, SOPs, and architecture documentation.
• Define and enforce best practices for cluster operations, deployments, and data pipelines.
• Contribute to continuous improvement initiatives and knowledge sharing.

Required Skills & Experience

•           8+ years of experience in Big Data / Data Platform Engineering in enterprise environments.

•           Strong hands-on experience with:

•           Hadoop ecosystem (HDFS, YARN, SPARK, MapReduce, HBase etc)

•           Apache Kafka (high-throughput environments)

•           OpenSearch/ Elasticsearch

•           Apache NiFi

•           AWS EMR +good knowledge of AWS Cloud.

•           Strong expertise in Linux system administration and scripting (Shell/Python)

•           Experience with Kerberos, data security, and access governance

•           Proven experience in handling high-volume, low-latency systems (preferably payments/trading) 

Work Schedule

• Work schedule is mainly focused to support Asia and EMEA (Paris) time zone; however, may have to support during non-office hours/weekends/public holidays for critical incidents or escalation as per the assigned on-call support requirements.

If interested, you can click on “Apply here” or write an e-mail to [email protected] with your updated resume.

NOTE: - Only shortlisted candidates will be contacted back.

Thanks & Regards

Deeksha Agarwal

EA Licence No.91C2918

Personnel Registration No.  R26161520