Data Engineer

Whitehall Resources are looking for a Data Engineer. This role is hybrid working with 2-3 days per week onsite in Cambridge, and the remainder remote working, for an initial 15 month contract.

***Inside IR35***

Responsibilities:

• Build highly scalable, fault-tolerant distributed data processing systems (batch and streaming) that handle tens of terabytes of data daily, supporting a petabyte-scale data lakehouse.
• Collaborate with Engineering stakeholders to own and deliver intelligence data products. Negotiate delivery milestones and build new data flows to ensure performance, scalability, and cost efficiency.
• Participate in architectural discussions, influence the product roadmap, and lead new initiatives from concept to delivery.
• Data quality, governance, compliance and observability across all stages of the data lifecycle.
• Serve as a mentor and data advocate, providing technical leadership and supporting team members within the business unit and organisation.

Required Skills and Experience:

• Degree and/or equivalent experience in engineering, computer science or a related data-intensive field.
• Proficient with one or more programming languages (Python, Scala).
• Hands-on experience with modern lakehouse technologies (e.g. Databricks, Spark, Kafka).
• Comfortable building pragmatic ETL/ELT workflows in AWS using orchestration frameworks or cloud-native tools.
• DataOps mentality. Understanding of how to apply automation, version control, testing, monitoring and continuous delivery to data systems.
• Excellent written and verbal communication skills, with the ability to collaborate effectively in fast-paced, diverse environments.
• Ability to present data and insights in an engaging and clear way, with strong attention to detail, and ensure data accuracy and quality.
• Solid understanding of data governance, security, and compliance frameworks.
• Experience with data visualisation (Tableau, Looker, Kibana or Power BI).
• Experience usingAI/ML data pipelines, MLOps, or related workflows and the processes around testing, monitoring, and SLAs.
• Passion for mentoring other engineers and contributing to the development of engineering standards.