Data Engineer

Duties/Responsibilities:

• Designing, building, and maintaining data pipelines to extract, transform, and load data from various sources (e.g. databases, web APIs, flat files) into a central data storage
• Implementing data quality checks and monitoring processes to ensure the accuracy and completeness of the data
• Collaborating with data analysts and data scientists to understand their data needs and to provide them with the data they need to do their jobs effectively
• Writing and maintaining documentation for data pipelines and processes
• Investigating and fixing issues with data pipelines and processes
• Participating in the design and implementation of data governance policies and procedures
• Keeping up-to-date with the latest developments and best practices in data engineering
• The ideal candidate for this role will have strong technical skills in programming (e.g. Python, SQL) and data processing technologies (e.g. Hadoop, Spark). They should also be able to work well in a team, be able to communicate effectively with both technical and nontechnical stakeholders, and be comfortable working in a fast-paced, constantly evolving environment.

Required Skills/Qualifications:

• A bachelor's or master's degree in a relevant field, such as computer science, data science, or engineering.
• Proficiency in programming languages such as Python. Experience with SQL and database technologies (e.g., MySQL, PostgreSQL) is also highly valuable.
• Experience with data processing technologies such as Hadoop, Spark, or Flink.
• Familiarity with data management and the ability to design and maintain data pipelines on Internal server.
• Strong problem-solving and analytical skills, with the ability to troubleshoot and debug issues with data pipelines.
• Good communication and collaboration skills, as the role often involves working closely with data analysts, data scientists, and other team members.

Preferred Qualifications:

• Master's degree in a related field
• Experience with machine learning techniques and tools
• Experience working in a business setting and using data to inform decision-making
• Minimum 5 years of hands-on experience in relevant field (Project or business use case)