Minimum Qualification
- Experience: 5-10 years of experience in data engineering,
- Strong exp in Snowflake.
- Programming & Scripting: Strong programming skills in Python and Linux Bash for automation and data workflows.
- Framework Proficiency: Hands-on experience with Luigi for orchestrating complex data workflows.
- Data Processing & Storage: Expertise in Hadoop ecosystem tools and managing SQL databases for data storage and query optimization.
- AWS Cloud Services: In-depth knowledge of AWS EC2, S3, RDS, and EMR to deploy and manage data solutions.
- Monitoring & Alerting Tools: Familiarity with monitoring solutions for real-time tracking and troubleshooting of data pipelines.
- Communication & Leadership: Proven ability to lead projects, communicate with stakeholders, and guide junior team members.
Additional experience
- Data Architecture: Experience designing or optimizing data lake solutions.
- Security Practices: Understanding of data security practices, data governance, and compliance for secure data processing.
- Automation & CI/CD: Familiarity with CI/CD tools to support automation of deployment and testing.
- Big Data Technologies: Knowledge of big data processing tools like Spark, Hive, or related AWS services.
- Advanced Analytics: Background in analytics or data science to contribute to more data-driven decision-making.
- Cross-Functional Collaboration: Experience collaborating with non-technical teams on business goals and technical solutions.