Hadoop Developer - Big Data

We are seeking a highly skilled and motivated Hadoop Developer to join our team and play a crucial role in developing and maintaining our big data processing pipelines.The Ideal Candidate :Experience :- 5+ years of experience in developing and maintaining big data applications using Hadoop ecosystem technologies.Technical Skills :- In-depth knowledge of Apache Hadoop architecture and core components (HDFS, YARN, MapReduce).- Proficiency in distributed data processing frameworks like Apache Spark (Spark SQL, Spark Streaming).- Expertise in data warehousing and querying using Apache Hive and HiveQL.- Strong understanding of Unix/Shell scripting for automating Hadoop tasks.- Solid programming skills in Python or Scala for developing big data applications.Additional Skills :- Experience with data ingestion tools like Sqoop, Flume, or Kafka (a plus).- Familiarity with data quality and data cleansing techniques.- Understanding of distributed file systems like HDFS and alternatives like Apache Parquet or Apache ORC.- Experience with big data visualization tools (e.g., Tableau, Power BI) is a plus.Soft Skills :- Excellent communication and collaboration skills to work effectively with cross-functional teams (data analysts, data scientists, etc.).- Strong problem-solving and analytical skills.- Ability to learn new technologies quickly and adapt to changing requirements.- A proactive and results-oriented approach to work.Responsibilities :- Design, develop, and implement big data processing pipelines using Hadoop and Spark.- Develop and maintain data extraction, transformation, and loading (ETL) processes using tools like Sqoop, Flume, or Kafka.- Write efficient MapReduce and Spark jobs to process large datasets.- Design and develop Hive tables and queries to store and analyze big data.- Automate Hadoop tasks using Unix/Shell scripting.- Troubleshoot and debug big data processing issues.- Monitor and optimize Hadoop cluster performance.- Work with data analysts and data scientists to understand data requirements and develop solutions.- Stay up-to-date with the latest advancements in big data technologies and best practices. (ref:hirist.tech)