Cloud Data Architect

Position : Cloud Data Architect

Purpose of the Position:

You will be a critical member of the InfoCepts Cloud Data Architect Team. This position requires a sound knowledge on Big Data using Cloud technologies, EMR, Athena, Spark-Scala, PySpark, S3, AWS Lambda etc. Strong foundation on database concepts and SQL is also required.

Location: Pune/Nagpur

Key Result Areas and Activities:

• Study existing technology landscape and understand current data integration framework and do impact assessment for the requirements
• Should be able to design complex Big Data use cases using AWS Services
• Should be able to reason about various architecture choices in terms of: Cost, Performance and Durability
• Should be able to suggest optimizations in existing designs
• Ensure optimal balance between cost and performance
• Project documentation, Adheres to Quality guidelines & Schedules
• Works hand in hand with PM for successful delivery of project and provide Estimation, scoping, scheduling assistance
• Effectively communicate architectural decisions to stakeholders
• Articulate design decisions very clearly with stakeholders
• Perform Proof-of-Concepts and document all observations before proposing a new solution.
• Conduct design review sessions with other teams and identify scope of improvement.
• Suggests automation to improve existing processes
• Assist other Data Engineers by providing expert advice or troubleshooting steps whenever required
• Keep suggesting new ways to improve team productivity
• Create technology focused training plans whenever required
• Deliver technology focused training sessions with team members whenever required
• Conduct Expert Knowledge Sharing sessions with Client Stakeholders whenever required
• Assist in RFPs for other Accounts whenever required
• Assist in designing case study documents whenever required

Work and Technical Experience:

Essential Skills:

• Overall, 9+ years of IT Exp
• 5+ years of AWS related project
• In depth knowledge of following AWS Services required: S3, EC2, EMR, Athena, AWS Glue, Lambda.
• Cloud Databases (Good to have) – AWS Aurora
• MPP Databases (anyone) – AWS Redshift, Snowflake, SingleStore
• Big Data (Must have) – Apache Spark
• Programming Language (Must have) – Scala, Python
• Big Data Table formats (Must have at least one) – Delta Lake (open source), Apache Iceberg
• Operating System (Must have) - Any flavor of Linux, Windows
• Must have very strong SQL Skills
• Orchestration Tools (Must have): Apache Airflow
• Expertise in developing ETL workflows comprising complex transformations like SCD, deduplications, aggregations etc.
• Should have thorough conceptual understanding of AWS VPC, Subnets, Security Groups, & Route Tables
• Should be a quick and self-learner and be ready to adapt to new AWS Services or new Big Data Technologies as and when required.
• Should know Data Fabric and Data Mesh concepts.
• Should know Data Warehousing concepts very well

Desirable Skills:

• Good to have cleared Associate Level or Professional Level AWS Certification
• Big Data (Good to Have) – Hadoop, Hive
• Programming Language (Good to have) – Java, Bash
• Big Data Platform (Good to have) – Databricks

Qualifications:

• Bachelor’s degree in computer science, engineering, or related field (Master’s degree is a plus)
• Demonstrated continued learning through one or more technical certifications or related methods.
• At least 10 years of IT experience; Minimum 5 years of experience on Cloud related projects

Qualities:

• Hold strong technical knowledge and experience
• Should have the capability to deep dive and research in various technical related fields
• Self-motivated and focused on delivering outcomes for a fast growing team and firm
• Able to communicate persuasively through speaking, writing, and client presentations
• Able to consult, write, and present persuasively
• Able to work in a self-organized and cross-functional team
• Able to iterate based on new information, peer reviews, and feedback
• Prior experience of working in a large media company would be added advantage