Asia Pacific Senior Data Engineer (Analytics)

Role Summary:

Our client is a global multinational
company, providing their customers with the data and tools to make more
informed business decisions. In this role, you’ll be working as an Analytics
Engineer within Analytics Center of Excellence on Experian’s internal Cloud
Platform. You’ll be managing data at scale and working on key initiatives such
as Data Productizations where we employ different big data technologies and
back/front ends to build solutions for our clients.

Responsibilities:

• Engage with
  clients and stakeholders to gather business and technical requirements, in
  order to create and maintain data pipeline (using Apache Spark or similar
  technologies), and workflow tools (Airflow or similar) for both Realtime and batch use cases.
  
• Develop backend
  APIs, support integrations and solutions deployment to cloud infrastructure
  platforms like AWS or Azure.
  
• Playing a key
  role in stakeholder communication and expectation management throughout the
  project lifecycle.
  

Requirements

• Degree or
  equivalent, with a high numeric content e.g. Mathematics, or Statistics, with at
  least 4 years of relevant experience across data engineering, ETL development,
  and cloud deployments.
  
• Experience in
  data warehouse technologies, data modelling and ETL development.
  
• Experience in a
  high-performing, large-scale technology driven environment.
  
• Exposure in
  creating/maintaining data pipeline and workflow tools for both Real-time and
  batch use cases. Cloud experience (AWS is preferred).
  
• Basic
  understanding of ML, Deep learning, Wrappers and APIs.
  
• Experience with
  Big Data ML toolkits, such as Mahout, SparkML, or H2O (Mandatory).
  
• Strong coding
  skill in SQL and (Python or Java or Scala).
  
• Exposure to
  Database architecture using RDBMS or NoSQL: Views/Tables, Disk usage and
  relational diagram.
  
• Excellent
  communication, interpersonal skills, and proven project management skills.
  
• Management of
  Hadoop cluster, with all included services, and ability to solve any ongoing
  issues with operating the cluster.
  
• Experience with
  building stream-processing systems, using solutions such as Storm or
  Spark-Streaming (Good to have).
  
• Good knowledge of
  Big Data querying tools, such as Pig, Hive, and Impala.
  
• Experience with
  integration of data from multiple data sources.
  
• Knowledge of
  various ETL techniques and frameworks, such as Flume.
  
• Experience with
  various messaging systems, such as Kafka or RabbitMQ.
  
• Good
  understanding of Lambda Architecture, along with its advantages and drawbacks.
  
• Experience with
  Cloudera/MapR/Hortonworks.