Data Platform Developer Data & Analytics · Lisbon · Fully Remote

Empowered by our motto, "making data matter," Keyrus stands at the forefront of global consultancy in Data & Analytics, AI & Machine Learning, and Digital Transformation. With a presence in 27 countries across 4 continents, we're on a mission to elevate the impact of data.

The significance of the Data Engineer position within the Data tower of the project cannot be overstated, as it plays a pivotal role in advancing the incorporation of Data Mesh principles and practices across the organization. Moreover, the role involves actively aiding the deployment of established and standard data architectures across diverse application teams, which are poised to evolve into specialized Data Domain teams.

In addition to these responsibilities, the role encompasses actively participating in the creation of Proof of Concepts for Data Mesh , while also offering expertise in leveraging contemporary technologies. This includes advising on the utilization of services available on public clouds or the development of in-house services on our internal Kubernetes based platform.

Focus of this position will be centered around technology that provides functionality in enabling a future Data Mesh.

How the day by day looks like in this job?

Data/Data Mesh Architecture:

• Design, develop, and maintain scalable data architectures, including databases, data lakes, and data warehouses.
• Implement best practices for data storage, retrieval, and processing.
• Drive the adoption of Data Mesh principles, promoting decentralized data ownership and architecture.
• Conceptualize, design, and implement Data Mesh Proof of Concepts (POCs) to validate decentralized data architectures.

Data Modeling:

• Design and implement data models supporting business requirements.
• Collaborate closely with data scientists and analysts to understand and structure data needs

Data Integration:

• Develop and maintain ETL (Extract, Transform, Load) processes for moving and transforming data from various sources into the data infrastructure.
• Ensure seamless integration of data from different systems and platforms.
• Establish and maintain ETL processes to integrate data into decentralized data products, aligning with Data Mesh principles.

Data Quality and Governance:

• Implement and enforce data quality standards and governance policies.
• Develop and maintain data documentation for metadata, lineage, and data dictionaries.

Platform Adaptation:

• Design and implement Kubernetes-based deployment strategies for scalable, reliable, and manageable data technologies.
• Collaborate with DevOps and Infrastructure teams to optimize data technology deployment processes within a Kubernetes environment.

Documentation:

• Document Data Mesh implementations, POC results, and best practices for knowledge sharing and future reference

For this job we need the following experience/skills:

• 5+ years of general IT experience
• 3+ years of Big Data experience
• Proven experience as a Data Engineer with a focus on designing and implementing scalable data architectures.
• Extensive experience in developing and maintaining databases, data lakes, and data warehouses.
• Hands-on experience with ETL processes and data integration from various sources.
• Familiarity with modern data technologies and cloud services.
• Proficient in designing and implementing data models to meet business requirements.
• Experience with Data Mesh
• Proficiency in data processing languages such as SQL, Java, Python or Scala

Knowledge and experience with at least some of the (Big) Data technologies/frameworks:

• Workflow orchestration (AirFlow/Oozie etc.)
• Data integration/Ingestion (Flume etc) .
• Messaging/Data Streaming (Kafka/RabbitMQ etc.)
• Data Processing (Spark, Flink etc.)
• RDBMS (PostgreSQL/MySql etc.)
• NoSQL Storages (MongoDB, Cassandra, Neo4j etc.)
• Timeseries (InfluxDB, OpenTSDB, TimescaleDB, Prometheus etc.)
• And/Or with their Cloud provided counterparts, i.e., Cloud Data/Analytics services (GCP, Azure, AWS)
• Familiarity with reference Big Data architectures (Warehouse, Data Lake, Data Lakehouse) and their implementation.
• GitLabBasic DevOps skills
• Agile knowledge

• Fluent in English (german would be a plus)
• Must be a Portugueseor Europeancitizen or have a work permit for Portugal

Nice-to-have technical knowledge (not mandatory):

• K8s Skills Container/Docker
• Helm
• Hands-on experience with K8s operators for Big Data technologies
• Cloud Native Technologies/Projects
• Terraform
• GitOps approach
• ArgoCD / Flux
• Familiarity with DevOps practices and optimization of data technology deployment processes.

What we offer at Keyrus Portugal:

• Salary based on your skills and experience and according to the Datamarket practices
• Meal allowanceof €8.32 per day
• Private Medical Insurance
• 22 days of annual leave
• Training Program through KLX (Keyrus Learning Experience)
  
  

Expected start date: as soon as possible

Why Keyrus?

Joining Keyrus means joining a market leader in the Data Intelligence field and an (inter)national player in Management Consultancy and Digital Experience.

You will be part of a young and ever learning enterprise with an established international network of thought leading professionals driven by bridging the gap between innovation and business. You get the opportunity to meet specialized and professional consultants in a multicultural ecosystem.

Keyrus gives you the opportunity to showcase your talents and potential, to build up experience through working with our clients, with the opportunity to grow depending on your capabilities and affinities, in a great working and dynamic atmosphere.