Requirements
Key responsibilities
• Data ingestion. Design and build repeatable pipelines for bulk and incremental ingestion from files, databases, APIs and other approved sources.
• Data standardisation. Profile, cleanse, deduplicate, transform and map heterogeneous records to an agreed information and metadata schema.
• Quality and control. Implement automated validation, reconciliation, exception management and measurable data-quality checks.
• Traceability. Maintain source-to-target mappings, provenance, ownership, lineage and auditable change records.
• Secure engineering. Apply least-privilege access, secure processing, controlled environments, testing and appropriate information handling.
• Integration and performance. Work with architects to optimise data stores, integration interfaces, load performance and reliable updates.
• Sustainable operations. Produce runbooks, monitoring guidance and documentation; transfer ingestion and support knowledge to operational teams.
Essential experience and skills
• Strong commercial experience in a senior data engineering role, delivering production-grade pipelines and integration solutions.
• Advanced SQL and practical Python or equivalent scripting capability; experience with ETL/ELT, APIs and structured/semi-structured data.
• Proven data profiling, mapping, deduplication, validation, reconciliation and exception-handling skills.
• Experience with relational and/or document data platforms, version control, automated testing and CI/CD practices.
• Ability to collaborate with information architects, solution architects, analysts and business stakeholders.
• Clear technical communication, documentation and delivery discipline in security-conscious environments.
Desirable experience
• Metadata catalogues, information/knowledge management platforms or enterprise search integration.
• Data-quality scorecards, lineage tooling and scalable orchestration/monitoring techniques.
• Experience working with formal information standards and assurance processes in regulated environments.
Expected contribution
An auditable ingestion capability, a reconciled and quality-controlled initial dataset, reusable validation controls, and documented processes that operational teams can maintain.
Your email job alert is now active!