Senior Data Engineer (Streaming + AI Systems)

We’re partnered with a capital-backed operator in the industrial / infrastructure space that’s investing heavily in building a modern data platform from the ground up. They’re hiring a Senior Data Engineer to lead the design and evolution of real-time data systems and scalable architecture supporting both operational intelligence and predictive modelling.

This is a high-ownership role at the intersection of data engineering, applied statistics, and production AI. You’ll operate as a senior individual contributor with end-to-end responsibility across pipelines, platform design, and ML enablement.

Why This Role Is Compelling

• True ownership of the data platform: architecture, pipelines, and production ML systems.
  




• Direct impact on operational performance and financial strategy, not just analytics.
  




• Senior-level autonomy with influence over technical direction and tooling choices.
  




• Lean, execution-focused team where senior engineers build and ship complete systems.
  




• Long-term stability paired with a builder environment and modern stack.
  

Scope and Responsibilities

• Architect and operate real-time and batch data pipelines (Kafka, Spark, AWS ecosystem).
  




• Lead the evolution of a scalable lakehouse architecture for analytics and ML workloads.
  




• Design and productionize end-to-end ML pipelines, including feature engineering and real-time inference.
  




• Apply statistical methods (time-series, regression, anomaly detection) to high-impact business problems.
  




• Partner with stakeholders across operations, finance, and product to translate complex requirements into robust systems.
  




• Own infrastructure design using IaC (Terraform or equivalent) to ensure reproducibility and scale.
  




• Drive performance, reliability, and observability across data systems.
  




• Evaluate and introduce new tools and patterns across data engineering and applied AI.
  

Requirements

Tech Environment

• AWS (Glue, Lambda, ECS, S3, and related services)
  




• Apache Spark + lakehouse frameworks (Delta or equivalent)
  




• Kafka / streaming architectures
  




• Terraform (Infrastructure as Code)
  

What They’re Looking For

• Proven experience building and scaling data platforms in production environments.
  




• Deep expertise in streaming systems and distributed data processing.
  




• Strong system design skills with the ability to operate at both architecture and implementation levels.
  




• Experience owning ML/data pipelines end-to-end, including production deployment.
  




• Solid grounding in statistical modelling and real-world applications.
  




• A bias toward ownership, pragmatism, and building systems that actually get used.