Hadoop Administrator

Job Title: Senior Software Engineer

 Experience: 8 – 11 years

 Location: SFO/ Houston

 
Primary Skills:

    Set up production Hadoop clusters with optimum configurations
    Drive automation of Hadoop deployments, cluster expansion and maintenance operations
    Manage Hadoop cluster, monitoring alerts and notification
    Deployment of upgrades, updates and patches
    Provide 24x7 tier-3 troubleshooting and break-fix support for production services
    Diagnosis of installation & configuration issues
    Diagnosis of cluster management issues
    Diagnosis of performance issues
    Job scheduling, monitoring, debugging and troubleshooting
    Monitoring and management of the cluster in all respects, notably availability, performance and security
    File system management and monitoring
    Data transfer between Hadoop and other data stores (incl. relational database)
    Ingesting additional data sources into Hadoop in either streaming or batch mode
    Commissioning and decommissioning worker nodes
    Configuring the cluster to be rack-aware
    Set up High Availability/Disaster Recovery environment
    Debug/Troubleshoot environment failures/downtime
    Working with data delivery teams to setup new Hadoop users. This job includes setting up Linux users, setting up Kerberos principals and testing HDFS, Hive, Pig and MapReduce access for the new users
    Set up security configurations in Hadoop
    Performance tuning of Hadoop clusters and Hadoop Map Reduce routines
    Manage and review Hadoop log files

Engineering Graduate, MCA, Masters/Post Graduates (preferably IT/ CS)