IT & Software

Lead Data Engineer

Lorien Resourcing

London · Greater London · United Kingdom

Overview

As Lead Data Engineer, you will drive the design, build, and operation of production-grade data platforms and AI-enabled solutions in an enterprise context. You’ll act as a senior technical anchor, delivering end-to-end data and AI capabilities while guiding others through examples and best practices. The role spans legacy to cloud-native environments, focusing on reliable systems, governance, and scalable data pipelines. You will partner with cross-functional teams to operationalize AI, improve data sharing, and enable secure, scalable analytics and ML workloads. This is a hands-on, impact-focused leadership role that shapes how data and AI are used safely and effectively at scale.

Responsibilities
  • Design, build, and enhance enterprise data platforms, pipelines, and services
  • Lead complex data engineering work end-to-end from problem definition to deployment and support
  • Develop cloud-native data platforms supporting analytics, ML, and AI workloads
  • Create data pipelines for traditional analytics and AI/ML use cases
  • Collaborate with stakeholders to translate problems into scalable data and AI solutions
  • Design data models, integrations, and storage for maintainability and reuse
  • Establish AI-led development frameworks including CI/CD, testing, governance, and deployment standards
  • Operationalize ML/AI solutions into production
  • Improve data sharing and onboarding of new data sources across teams
  • Diagnose and resolve complex data/platform issues in enterprise environments
  • Contribute to an engineering culture focused on quality and continuous improvement
Key requirements
  • Significant hands-on experience as a Senior or Lead Data Engineer on complex, enterprise-scale systems
  • Strong Python and Spark skills with production data pipelines
  • Deep understanding of the full data engineering lifecycle (ingestion, transformation, storage, serving, reuse)
  • Experience designing integrations across diverse data sources and legacy environments
  • Experience building and supporting cloud-based data platforms, preferably AWS
  • Proven ability to design scalable data models and architectures
  • Solid understanding of data governance, security, compliance, and operational controls
  • Experience with modern engineering practices: source control, automated testing, CI/CD, infrastructure as code, deployment automation
  • Ability to communicate with both technical and non-technical stakeholders
  • Experience supporting ML/AI workloads through production-grade data engineering solutions
  • Excellent communication with technical and non-technical stakeholders
  • Delivery-focused mindset and collaboration
  • Problem-solving mindset under complex enterprise conditions
  • Python
  • Spark
  • AWS data services (S3, Glue, EMR, Redshift, Athena, Lambda, SageMaker, Bedrock)

Reference: WJ-747_30993621

Apply now

Continue on the employer's official application - the same link they use for every candidate.

More jobs

Find more on GigBlows

This role is listed on GigBlows for discovery and search. Hiring decisions and applications are handled by the employer or their chosen application system.