Data Engineer

Location: Remote, UK, with travel to London 1-2 times per quarter

Duration: 6 months initially

Start Date: October 2026

We are looking for an experienced Data Engineer to support a major Trading Automation programme.

You will design and build modern data pipelines and platform capabilities using Databricks and Azure, providing the trusted and scalable data foundations required for AI-driven trading automation.

Key Responsibilities

• Design, build and maintain scalable data pipelines using Databricks and Azure.
• Develop ingestion and transformation processes for structured and unstructured data.
• Build reliable data products to support trading workflows and Generative AI applications.
• Implement data architectures using Delta Lake and lakehouse principles.
• Apply data quality, governance, lineage and security controls.
• Optimise pipelines and datasets for performance, reliability and scalability.
• Collaborate with AI engineers, trading teams and business stakeholders.
• Support automated deployment, monitoring and operational management of data workloads.
• Advanced experience with Databricks, Apache Spark and PySpark.
• Strong knowledge of Azure data services.
• Experience building batch and streaming data pipelines.
• Strong Python and SQL skills.
• Experience with Delta Lake and lakehouse architecture.
• Knowledge of data modelling, testing, quality and governance.
• Experience with CI/CD, Git and Infrastructure as Code.
• Comfortable working independently in a remote, client-facing environment.
• Experience within Energy Trading, commodities, Capital Markets or Financial Services.
• Experience supporting AI, machine learning or Generative AI workloads.
• Knowledge of Databricks Unity Catalog and MLflow.