Data Engineer

The Data Engineer will architect, build, and continuously enhance enterprise-grade data infrastructure and pipelines, ensuring the delivery of high-quality, trusted, and scalable data that underpins effective decision-making across SolarAfrica’s expanding portfolio of IPP plants, off-takers, and commercial partners.

Position of the job in the organisation:

The Data Engineer will report directly to the Head of IT.

Principal responsibilities:

• Architect and implement scalable ELT/ETL pipelines on AWS and/or Azure, leveraging cloud-native services (e.g., AWS Glue/S3/Redshift, Azure Data Factory/ADLS/Synapse) and Databricks where applicable


• Integrate and transform data from Financial, ERP (Dynamics 365), CRM, Sales, and operational systems into a unified analytical data platform


• Implement and enforce Medallion Architecture (Bronze → Silver → Gold) patterns, ensuring clean separation of raw ingestion, cleansed/conformed, and business-ready data layers


• Develop rigorous pipeline testing frameworks including unit tests, data reconciliation, schema validation, row count checks, and end-to-end UAT coordination with business stakeholders


• Implement data quality monitoring, alerting, and governance controls — maintaining full lineage and documentation across all data assets


• Build and maintain Power BI semantic models and enterprise reports for operational, investor, and executive reporting


• Optimise cloud resource usage to ensure solutions are cost-effective — including compute right-sizing, scheduling optimisation, storage tiering, and active cloud spend monitoring


• Engage independently with internal and external stakeholders to translate business requirements into technical data solutions, managing delivery from scoping through to production deployment


• Maintain data documentation, including pipeline runbooks, schema registries, and architectural decision records (ADRs)


• Any other job duties associated with this role.

Experience & Skills

• Bachelor’s degree in Computer Science, Information Systems, Engineering, Mathematics, or a related field


• 5–8 years of hands-on Data Engineering experience with demonstrable ownership of senior-level projects


• Strong SQL skills, including complex query optimisation, window functions, and performance tuning on large datasets


• Proven experience building production-grade ELT/ETL pipelines using Python (PySpark preferred)


• Hands-on experience with AWS (Glue, S3, Redshift, Lambda, Step Functions) and/or Azure (ADF, ADLS Gen2, Synapse Analytics, Azure Databricks)


• Experience implementing Medallion Architecture (Bronze/Silver/Gold) in a cloud lakehouse environment


• Proficiency in Power BI, including Power Query, DAX modelling, semantic layer design, and publishing enterprise-grade reports


• Demonstrated experience integrating ERP (Dynamics 365 preferred), CRM, and Financial/Sales data platforms into analytical environments


• Strong understanding of data governance, data quality frameworks, and access control best practices


• Experience with data warehousing solutions (Redshift, Synapse, Snowflake, or equivalent)


• Ability to produce architectural documentation: ERDs, data flow diagrams, source-to-target mappings


• Previous experience with ERP (preferably Dynamics 365)

Strong Differentiators:

• Hands-on experience with Databricks (Delta Lake, Unity Catalog, PySpark notebooks, Databricks Workflows)


• Experience in renewable energy, IPP, asset management, or infrastructure environments

Specific attributes:

• An architectural thinker who approaches new integrations with a design-first mentality before writing code


• Applies test-driven practices to data pipelines, not satisfied until data accuracy is independently verified


• Proactively manages cloud costs and flags inefficiencies without being prompted


• Can operate fully independently in an ambiguous environment, self-managing priorities across concurrent projects


• Communicates technical concepts clearly to non-technical stakeholders (finance, operations, executives)

Desirable attributes:

• Familiarity with Prefect 3 for orchestrating data ingestion workflows.


• Exposure to creating monitoring dashboards/alerts in Grafana Cloud

What Does Success Look Like?

By month 3 

• I will understand all core data sources, pipelines, and dependencies


• I will have documented existing data architecture and known data issues


• I can independently maintain and troubleshoot existing data pipelines


• I will be able to engage confidently with stakeholders on data requirements

By month 6 

• I will have improved data reliability, accuracy, and pipeline stability


• I can design and deploy new data pipelines with minimal oversight


• I will have implemented basic data quality checks and monitoring


• I will be able to support analysts and business users with trusted datasets

By month 12 

• I will have contributed to scalable, future-ready data architecture


• I can proactively identify and resolve data risks before they impact the business


• I will have reduced data-related incidents and rework


• I will be able to enable faster, better decision-making through reliable data

Core Values:

We hire, reward, and recognise our team against these values. It is imperative that you believe in these values and demonstrate them consistently.

• We are passionate and proud of what we do.


• We communicate candidly, especially when it is difficult


• We take the initiative, share our mistakes, and grow together


• We are dependable and take accountability


• No one person is bigger than the solution - no egos.

We are currently only considering candidates based in South Africa with valid working rights. Unfortunately, we are unable to consider applications from outside the country at this time.