Senior Data Engineer

In this role you will design, build and maintain modern cloud-based data platforms to unlock data value for government, defence and regulated sectors. You’ll work with cross-functional teams to deliver reliable data pipelines and analytics capabilities that support critical services. The role combines data engineering, software development and cloud technologies to solve real-world problems, with opportunities to grow technical and leadership skills. You will contribute to scalable, quality-driven data solutions that enable informed decisions and measurable outcomes. This is a mission-driven opportunity to advance data capabilities across diverse client environments.

• flexible working arrangements
• development and certifications
• mentoring and knowledge sharing
• exposure to diverse projects across industries
• pension
• holiday allowance

• Design, build and maintain ETL/ELT data pipelines using AWS or Azure cloud-native services
• Develop and support data integration components and Java-based microservices where required
• Prepare, transform and validate structured and semi-structured data (parquet, avro, JSON)
• Write SQL to query and optimise data on cloud analytical platforms (Athena, Redshift, Synapse)
• Build scalable data processing solutions using PySpark, Python and cloud orchestration
• Implement data quality checks, monitoring and logging for trusted data delivery
• Support performance tuning and optimisation of pipelines, SQL workloads and integration services
• Collaborate with data architects, analysts, software engineers and client stakeholders to deliver solutions
• Support DevOps practices with source control, automated testing and CI/CD
• Communicate progress, risks and technical decisions to technical and non-technical stakeholders

• AWS Glue, Lambda, Step Functions or Azure Data Factory, Synapse Pipelines, Azure Functions and Logic Apps
• PySpark
• SQL (Athena, Redshift, Azure Synapse SQL)
• AWS S3 or Azure Data Lake Storage Gen2
• Python and/or Java
• ETL/ELT data pipeline development
• Data quality, validation and monitoring
• Git, CI/CD and Agile delivery practices
• Experience being most Senior of Lead within a project
• Eligibility for SC Clearance

• Collaboración y trabajo en equipo
• Comunicación eficaz de progresos y riesgos
• Colaboración con múltiples partes interesadas
• PySpark
• SQL (Athena, Redshift, Synapse)
• AWS Glue, Lambda, Step Functions