IT & Software

Lead/Principle Data Engineer

Sanderson Recruitment

Cheltenham · Gloucestershire · United Kingdom

Overview

You will design, build, and optimise secure data pipelines for large-scale government data platforms. In an Agile, cross-functional team, you’ll deliver batch and real-time processing solutions that support critical decision-making. This role emphasises secure environments, modern cloud tech, and open-source tools to drive reliable data delivery and platform evolution. You’ll work closely with engineers and stakeholders to ensure data quality, performance, and compliance, contributing to mission-critical outcomes.

Responsibilities
  • Develop, test, and maintain robust data pipelines meeting business requirements
  • Design and optimise batch and real-time data processing solutions
  • Contribute to logical/physical database design across data platforms
  • Enhance platform capabilities to tackle new data challenges
  • Analyse complex datasets to improve processing, quality, and performance
  • Ensure data models align with architecture standards and definitions
  • Perform root cause analysis on data issues and implement solutions
  • Collaborate with engineers, architects, and stakeholders to deliver scalable data solutions
  • Support continuous improvement and best practices in data engineering and platform development
  • Produce technical documentation for pipelines and system enhancements
Key requirements
  • Experience building and maintaining data pipelines with modern data processing frameworks
  • Experience with Agile delivery in fast-paced environments
  • Strong data modelling knowledge and understanding of data structures
  • Experience with AWS (EC2, RDS, Lambda, Athena, Glue) applications
  • Relational and NoSQL database experience with query tuning and optimization
  • Proficiency in Python
  • Experience applying Test-Driven Development (TDD)
  • Strong analytical and problem-solving abilities
  • Excellent communication and cross-functional collaboration
  • Bachelor’s degree or equivalent experience
  • Collaborative mindset
  • Strong communication
  • Analytical thinking
  • AWS services (EC2, RDS, Lambda, Athena, Glue)
  • Data pipelines and processing frameworks
  • Data modelling and database design

Reference: WJ-747_30862641

Apply now

Continue on the employer's official application - the same link they use for every candidate.

More jobs

This role is listed on GigBlows for discovery and search. Hiring decisions and applications are handled by the employer or their chosen application system.