IT & Software

Senior Data Engineer

Luxoft

London · Greater London · United Kingdom

Overview

In this senior role, you will architect and lead the migration of a large-scale ETL estate from on-prem tools to AWS. You will liaise with client engineers to reverse-engineer the current landscape and define the target state architecture across data processing, workflows, and operating models. You’ll design reusable migration patterns, validate inventories, and run a proof-of-concept to de-risk the approach. This engagement emphasizes data engineering excellence, architectural leadership, and delivery planning within a Tier 1 financial services environment.

Responsibilities
  • Design, build, and maintain ETL/ELT pipelines on AWS
  • Migrate existing data pipelines and workloads from legacy/on-prem systems to AWS
  • Develop and optimize data models for data warehousing (Redshift, data lake on S3)
  • Build and orchestrate data workflows (Glue, Step Functions, Lambda, EMR)
  • Implement data quality checks, validation, and reconciliation processes
  • Optimize pipeline performance and manage compute/storage costs
  • Ensure data security and access control (IAM, KMS, encryption)
  • Monitor and troubleshoot pipeline failures (CloudWatch, logging)
  • Collaborate with data analysts, BI developers, and architects on data requirements
  • Document data pipelines, architecture, and operational runbooks
Key requirements
  • 5+ years of experience
  • Hands-on experience building data pipelines on AWS (Glue, EMR, Lambda, Step Functions)
  • Strong SQL and experience with data warehousing (Redshift, dimensional modeling)
  • Proficiency in Python or Scala for data engineering
  • Experience with S3-based data lake architecture (partitioning, cataloging, formats like Parquet)
  • Experience migrating data pipelines from on-prem or other cloud platforms
  • Understanding of data governance, security, and access control on AWS
  • Experience with CI/CD for data pipelines
  • collaboration with cross-functional teams
  • ability to influence stakeholders
  • structured delivery planning
  • AWS data engineering stack (Glue, EMR, Lambda, Step Functions)
  • SQL and data warehousing (Redshift)
  • Python or Scala for data engineering

Reference: WJ-747_30156031

Apply now

Continue on the employer's official application - the same link they use for every candidate.

More jobs

This role is listed on GigBlows for discovery and search. Hiring decisions and applications are handled by the employer or their chosen application system.