Lead Data Architect

TECH LEAD – ENTERPRISE DATA & AI PLATFORM - LONDON

We’re recruiting for a senior, hands-on Technology Lead to build a new enterprise data and AI platform for a major global commodities trading organisation.

The successful candidate will have personally built a comparable data/AI platform before, not just directed one, and will bring that experience to bear leading a team of internal technology leads while directing the work of multiple external vendors and delivery partners

You’ll own the technical delivery of the platform while remaining close to the architecture, engineering and code. The environment will connect data across trading, risk, finance, operations and asset systems, creating a scalable foundation for analytics, streaming and AI use cases.

The role

You’ll take technical ownership of the platform from its initial foundations through to production delivery and wider adoption.

Key responsibilities include:

• Building the core AWS and Databricks platform
• Designing the lakehouse architecture using Delta and/or Iceberg
• Delivering scalable ingestion, transformation and data-product pipelines
• Building real-time streaming and CDC capability using technologies such as Kafka, Kinesis or Flink
• Developing the semantic, catalogue, lineage and knowledge-graph layers
• Supporting governed AI, agentic and retrieval-based capabilities
• Establishing engineering standards across CI/CD, testing, observability, GitOps and infrastructure as code
• Leading internal technical teams and providing direction to external vendors and delivery partners
• Owning technical design decisions, delivery sequencing and platform reliability
• Enabling domain teams to build and release data products independently

What we’re looking for

• Extensive experience designing and building enterprise-scale data or AI platforms
• Previous ownership of a comparable platform build from the ground up
• Strong hands-on knowledge of AWS and Databricks
• Experience with Unity Catalog, Delta Lake, Iceberg and streaming workloads
• Production experience with Kafka, Kinesis, Flink or comparable streaming technologies
• Knowledge of ClickHouse or another low-latency analytical database
• Experience designing or implementing knowledge graphs using Neo4j, Stardog or similar technology
• Strong understanding of hybrid cloud and on-premise integration
• Experience establishing robust SDLC, CI/CD, testing and infrastructure-as-code practices
• A track record of leading technical teams and managing third-party delivery partners

Experience within commodities, energy trading or financial services would be highly advantageous, particularly where this includes trading, position, P&L, exposure or market-data environments.

This role would suit someone who is still a builder at heart: technically credible at architecture and code level, comfortable leading several delivery streams and able to turn a long-term platform vision into working, production-ready releases.