IT & Software

Data Engineer

Scottish and Southern Energy

Perth · Perth and Kinross · United Kingdom

Overview

In this role you will design and operate data ingestion, transformation and serving layers on Azure Databricks to support reliable data products and insights. You will help build the Renewables Data Platform and migrate solutions from Synapse, working with data architects, analysts and product teams to deliver scalable ELT/ETL pipelines with a focus on quality, security and performance. This hands-on position emphasizes collaboration, modern data modeling and governance to enable data-driven decisions in a renewables context. You will work in a cross-functional Data & AI team to impact business outcomes.

Pay / Benefits
  • private healthcare discounts
  • gym membership discounts
  • free online GP
  • 24/7 counselling
  • interest-free loans for tech and transport
  • Cycle to Work scheme
Responsibilities
  • Build and maintain metadata-driven pipelines in Azure Databricks for batch and streaming data
  • Develop Delta Lake layers (bronze/silver/gold) for BI/analytics and Power BI integration via Databricks SQL
  • Perform large-scale transformations with Databricks pipelines, PySpark and SQL, ensuring data quality and performance
  • Implement data quality, monitoring and governance practices into pipelines and data products
  • Contribute to Governance, Security & Compliance using Unity Catalog lineage and code-managed permissions
  • Support CI/CD and version-controlled solutions in collaboration with DevOps and product teams
  • Migrate and modernize data solutions from Synapse as part of renewing the platform
  • Collaborate with stakeholders to refine requirements and ensure solutions meet business needs
  • Champion data protection controls and contribute to SLAs, SLOs and data contracts
  • Continuously develop skills with evolving tools and technologies
Key requirements
  • Proven data engineering experience with batch and streaming pipelines
  • Experience with Azure Databricks, PySpark and SQL
  • Delta Lake architecture knowledge (bronze/silver/gold)
  • Power BI integration and Databricks SQL experience
  • Strong communication with technical and non-technical stakeholders
  • Ability to manage workload in a growing business
  • Awareness of governance, security and data protection practices
  • CI/CD and version-controlled development experience
  • communication
  • adaptability
  • proactive mindset
  • Azure Databricks
  • PySpark
  • SQL

Reference: WJ-747_31159574

Find more on GigBlows

This role is listed on GigBlows for discovery and search. Hiring decisions and applications are handled by the employer or their chosen application system.