Data Engineer (Databricks)

Contract: Initial 6-month contract (long-term transformation programme)

X4 Technology are partnered with a leading global energy trading organisation on a major tech transformation - migrating from a legacy ETRM platform onto Endur, and modernising its enterprise data platform with Azure Databricks.

As part of this multi-year programme, Oracle-based data platforms are being rebuilt into a scalable, cloud-native Azure Databricks platform supporting Endur, global reporting, analytics and risk - cutting critical data latency from 10–15 minutes down to near real-time.

They're after an experienced Data Engineer to help design and build that platform hands-on - combining deep engineering with real technical ownership, shaping standards, performance and future architecture along the way.

Responsibilities of the Data Engineer (Azure Databricks)

• Design, build and optimise scalable Azure Databricks pipelines using PySpark, SQL, Delta Lake and Structured Streaming
• Lead the migration of legacy Oracle platforms into Azure Databricks, delivering high-performance solutions for trading, reporting and analytics
• Drive engineering best practice across performance, scalability, CI/CD, testing, governance and cloud cost
• Work closely with architects, platform teams and stakeholders to shape solutions, mentor engineers and influence platform direction

Requirements for the Data Engineer (Azure Databricks)

• Extensive hands-on experience delivering enterprise-scale Azure Databricks platforms - strong PySpark, Python, SQL, Delta Lake and Structured Streaming (Delta Live Tables a bonus)
• Strong grasp of modern cloud data architecture, modelling, performance optimisation, governance and best practice
• Experience on large-scale cloud migration or platform modernisation programmes, with the ability to influence direction and mentor others
• Energy trading, commodities, capital markets or financial services background - a plus, not a must

A brilliant chance to work on one of the biggest Azure Databricks transformations in energy trading, helping build a modern cloud platform underpinning trading, risk, reporting and analytics - with real influence over technical decisions while staying hands-on in a high-performing team.

Interviews moving fast, with a view to onboarding ASAP.