Senior Data Engineer

In this Senior Data Engineer role at a fast-growing UK-based software business, you will build scalable data solutions in Databricks to establish a single source of truth and empower trusted self-service analytics. You will own end-to-end data pipelines and collaborate across Finance, Product, Sales, Marketing and Engineering to translate business needs into technical solutions. You will shape the enterprise data architecture with a focus on scalability, security and performance, and embed data quality and governance across the data lifecycle. You will mentor a junior engineer and contribute to maturing a growing data function, gaining visibility into data strategy and platform evolution.

• health insurance
• income protection
• life assurance
• subsidised gym membership
• 25 days' holiday
• birthday off

• Design, build and maintain scalable data pipelines and data products using Databricks, PySpark, Spark SQL and Delta Lake
• Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems
• Shape enterprise data architecture, balancing scalability, performance, security and maintainability
• Partner with stakeholders across Finance, Product, Sales, Marketing and Engineering to translate business challenges into technical solutions
• Embed data quality, governance and access controls throughout the data lifecycle
• Mentor a junior engineer and help mature a growing data function

• Senior level experience in data engineering with strong Databricks skills (PySpark, Spark SQL, Delta Lake)
• Advanced SQL and Python coding skills
• Track record of building and optimising enterprise-scale data pipelines and ETL/ELT solutions
• Solid grounding in data architecture, modelling and modern data platform design
• Cloud experience, preferably AWS (Azure also acceptable), and familiarity with CI/CD and source control best practice

• Stakeholder collaboration
• Mentoring and coaching
• Problem solving and analytical thinking
• Databricks
• PySpark
• Spark SQL