Tasks
- Collaborate with stakeholders to understand business requirements and translate them into data engineering solutions.
- Design and oversee the overall data architecture and infrastructure, ensuring scalability, performance, security, maintainability, and adherence to industry best practices.
- Define data models and data schemas to meet business needs, considering factors such as data volume, velocity, variety, and veracity.
- Select and integrate appropriate data technologies and tools, such as databases, data lakes, data warehouses, and big data frameworks, to support data processing and analysis.
- Ensure that data engineering solutions align with the organization's long-term data strategy and goals.
- Evaluate and recommend data governance strategies and practices, including data privacy, security, and compliance measures.
- Collaborate with data scientists, analysts, and other stakeholders to define data requirements and enable effective data analysis and reporting.
- Provide technical guidance and expertise to data engineering teams, promoting best practices, and ensuring high-quality deliverables. Support to team throughout the implementation process, answering questions and addressing issues as they arise.
- Oversee the implementation of the solution, ensuring that it is implemented according to the design documents and technical specifications.
- Stay updated with emerging trends and technologies in data engineering, recommending, and implementing innovative solutions as appropriate.
- Conduct performance analysis and optimization of data engineering systems, identifying and resolving bottlenecks and inefficiencies.
- Ensure data quality and integrity throughout the data engineering processes, implementing appropriate validation and monitoring mechanisms.
- Collaborate with cross-functional teams to integrate data engineering solutions with other systems and applications.
- Participate in project planning and estimation, providing technical insights and recommendations.
- Document data architecture, infrastructure, and design decisions, ensuring clear and up-to-date documentation for implementation, reference, and knowledge sharing.
Requirements
What We're looking for:
- At least 6 years of experience as Data Architect, including min. 4 years of experience working with GCP cloud-based infrastructure & systems.
- Deep knowledge of Google Cloud Platform and cloud computing services.
- Strong experience in the Data & Analytics area.
- Strong understanding of data engineering concepts, including data modeling, ETL processes, data pipelines, and data governance.
- Expertise in designing and implementing scalable and efficient data processing frameworks.
- In-depth knowledge of various data technologies and tools, such as columnar databases, relational databases, NoSQL databases, data lakes, data warehouses, and big data frameworks.
- Knowledge of modern data transformation tools (such as DBT, Dataform).
- Knowledge of at least one orchestration and scheduling tool.
- Programming skills (SQL, Python, other scripting).
- Tools knowledge: Git, Jira, Confluence, etc.
- Experience in selecting and integrating appropriate technologies to meet business requirements and long-term data strategy.
- Ability to work closely with stakeholders to understand business needs and translate them into data engineering solutions.
- Strong analytical and problem-solving skills, with the ability to identify and address complex data engineering challenges.
- Knowledge of data governance principles and best practices, including data privacy and security regulations.
- Excellent communication and collaboration skills, with the ability to effectively communicate technical concepts to non-technical stakeholders.
- Experience in leading and mentoring data engineering teams, providing guidance and technical expertise.
- Familiarity with agile methodologies and experience in working in agile development environments.
- Continuous learning mindset, staying updated with the latest advancements and trends in data engineering and related technologies.
- Strong project management skills, with the ability to prioritize tasks, manage timelines, and deliver high-quality results within designated deadlines.
- Strong understanding of distributed computing principles, including parallel processing, data partitioning, and fault-tolerance.
What Will Set You Apart:
- Certifications in big data technologies or/and cloud platforms.
- Experience with BI solutions (e.g. Looker, Power BI, Tableau).
- Experience with ETL tools: e.g. Talend, Alteryx
- Experience with Apache Spark, especially in GCP environment.
- Experience with Databricks.
- Experience with Azure cloud-based infrastructure & systems.
Missing one or two of these qualifications? We still want to hear from you! If you bring a positive mindset, we'll provide an environment where you feel valued and empowered to learn and grow.