Senior Data Engineer

Role Overview:

An exciting data business is seeking a dynamic and entrepreneurial individual to operate its consumer data environment and service its data requirements to deliver on the business’ client and operational needs. The role will enable the individual to become the owner of a complex and exciting data environment within a bigger group of data businesses.

They would need to work closely with the client management team, data scientists, analysts, and other stakeholders to understand data requirements and ensure the availability, quality, and accessibility of the data.

Being a young company that is part of a group that is mature and experienced in data management, they will also need to work closely with the group data function to provide input and seek guidance to enable a consistent approach to how data is managed across the group and form part of the larger group data function.

The role will have a direct reporting line into the Business Unit (MD/COO) on guidance on work and prioritization and a dotted reporting line into the Group Head of Data on guidance on technical and architectural guidelines.

Key Responsibilities:

Design and own all aspects of data engineering from delivery planning, estimating and analysis all the way through to data architecture and pipeline design, delivery, and production implementation. Construct end to end data service solutions.

Data Pipeline Development: Designing, implementing, and optimizing scalable and efficient data pipelines to extract, transform, and load (ETL) data from various sources into data storage systems. This may also require engagement and collaboration with other data teams from whom data is sourced.

Data Integration: Integrating data sources and systems, ensuring seamless data flow and compatibility between different environments. Design and implementation of complex data solutions ranging from batch to hybrid landscapes. Employ a variety of languages and tools (e.g. scripting languages) to marry systems together.

Data Quality Assurance: Implementing data quality checks, data validation processes, and data cleansing techniques to ensure the accuracy, consistency, and integrity of data.

Data Monitoring and Troubleshooting: Proactively monitoring data processes and systems, identifying and resolving data-related issues, and optimizing performance bottlenecks. Recommend ways to improve data reliability, efficiency and quality.

Operating and Maintaining Data Tools: Assess and improve data tools that have been built within the business and operate these data tools as and when needed. Automate data processes end to end (acquire, ingest, transform, model, store)

Collaboration and Documentation: Collaborating with cross-functional teams, such as data scientists, analysts, and software engineers, to understand data requirements and deliver efficient data solutions. Documenting data processes and configurations for knowledge sharing and future reference. Contribute towards a team that develops, constructs, tests and maintains architectures (such as databases and processing systems) and ensure data architecture will support the stakeholder requirements. Liaise and interface with stakeholders in technical advisory to support and enhancement services.

Emerging Technologies: Staying updated with industry trends and emerging technologies related to data engineering, recommending innovative solutions to improve data management and analytics capabilities.

Qualification & Experience:

• Matric (Grade 12)
• Applicable Diploma or Degree
• 10 years working in/across Data Engineering, Analysis & Data Warehousing
• 10 years’ experience with SQL and SSIS packages

Technical Skills:

• Proficient in Microsoft SQL Server
• Strong proficiency in TSQL programming (stored procedures, user defined functions, queries, triggers)
• Constructing data queries by combining multiple data sources.
• Knowledge of data integration techniques and tools, such SSIS for ETL/ELT processes, APIs, and data streaming an advantage
• Data Cleaning / Integrity Checking
• Proficiency in data modelling and database design of relational databases.
• Iterative Testing including debugging and refactoring.
• Understanding of data governance, security, and privacy principles.
• Experience working in a multi-disciplinary team where your own responsibility included transforming data into clean, organized data structures with the emphasis of design supporting analysis and construction of data products
• Ability to be self-sufficient from a technical point of view
• Experience of creating reports using Excel or equivalent.
• Experience with data visualization in Power BI and similar tools
• It will also be an advantage to have:

- Some experience in a programming language (such as Python/R).

- Data Architecture experience

- Data Management and Governance

- Experience of consuming APIs

- Data Warehousing automation

- Experience with consumer and retail data

Non-technical Skills:

• Strong problem-solving and analytical skills, with attention to detail. (Core to the role)
• Excellent communication and teamwork abilities, especially with the ability to effectively communicate with non-technical people • Enjoy supporting technical implementations.
• Able to build strong networks and relationships, internally and with customers.
• Can effectively multi-task and manage conflicting priorities, coupled with good attention to detail.
• Aptitude for providing timely and accurate responses to client inquiries and managing client expectations.
• A passion to learn and extend knowledge outside of the work sphere.
• Capable of identifying, embracing and initiating change in an agile and fast paced environment.
• The individual must display the characteristics of a self-starter. As such be an individual who takes initiative and demonstrates a proactive and motivated approach to their work. They should be driven by intrinsic motivation and must not rely heavily on external direction or supervision to get things done.
• Key characteristics include:

Initiative: should be someone who takes the initiative to identify and pursue opportunities or tasks without needing to be explicitly told or instructed.

Motivation: Should be internally motivated and have a strong drive to achieve their goals and deliver results. They set high standards for themselves and constantly seek self-improvement.

Resourcefulness: Should be a resourceful individual who is able to find solutions to problems independently. Should be proactive in seeking out information, tools, and resources to overcome challenges and achieve objectives.

Independence: Can work effectively with minimal supervision or guidance. They are self-reliant and have the ability to make decisions and take responsibility for their actions.

Proactivity: Be proactive in identifying opportunities, anticipating potential issues, and taking action to address them.

Time Management: Should have good time management skills and capable of prioritizing tasks and managing workload effectively. Should be capable to set goals, create plans, and meet deadlines without constant reminders or supervision.

Adaptability: Be adaptable and flexible in approach. Quickly adjust to changing circumstances, learn new skills, and embrace challenges with a positive mindset.

Persistence: Should demonstrate perseverance and determination when faced with obstacles or setbacks. They maintain a positive attitude and remain focused on their goals, finding alternative solutions if needed.

What we offer:

• The opportunity to work with an experienced data team across the group and within a group of proven successful data business in South Africa.
• An opportunity to be responsible for a small data operations unit
• To be part of a South African group that has expanded globally
• A leadership team that focusses on a care and growth style to leadership and cares deeply for their people
• An opportunity to be part of a cutting-edge data business
• Hybrid working model

Company Overview:

Lightstone Retail is high-growth data-tech business that designs and builds datadriven solutions for the Retail and Consumer facing Industry. We combine geospatial data and consumer insights, advanced analytics and deep market insights to help our clients make better location, strategy and marketing decisions.

Our primary clients are in retail, media, telecommunications and commercial property.

We are the youngest company in the well-known Lightstone stable with a defined track record and ambitious growth prospects to propel the group forward with a 3x growth plan over the next 4 years. Lightstone is a well-known brand in building data solutions that solve industry needs across the property and Automotive industries, with Retail as a key growth driver for the future.

We set high standards and hire exceptional people and provide them with the environment where they can excel. Our non-corporate culture encourages flexibility, collaboration and innovation.