Principal Azure Data Engineer (Databricks)
Capco
Overview
As Principal Azure Data Engineer (Databricks) you will architect and lead enterprise-grade data solutions on Databricks within Azure. You’ll guide end-to-end delivery, ensure secure, scalable pipelines, and uphold engineering best practices across projects. Collaborating with clients and cross-functional teams, you’ll shape data strategy and governance while mentoring engineers. This role offers the chance to influence data maturity for Tier 1 financial institutions in a collaborative, growth-focused environment.
Pay / Benefits- Discretionary bonus
- Competitive pension
- Health insurance
- Life insurance
- Mental health support
- Family-friendly leave
- Lead end-to-end delivery of Databricks-based solutions on Azure
- Architect secure, scalable pipelines with DeltaLake, Spark Structured Streaming, and Unity Catalog
- Develop and enforce data engineering best practices across the data lifecycle
- Engage with clients to define solution strategies and data governance frameworks
- Mentor engineering teams and contribute to internal capability development initiatives
- Databricks platform expertise including Unity Catalog, DeltaLake, and orchestration
- Python, PySpark, and distributed data processing experience
- Extensive CI/CD experience with Azure DevOps, Jenkins, GitHub Actions
- Deep understanding of data lakehouse concepts, data modeling, GDPR-compliant design
- Experience building robust, production-grade data pipelines from ingestion to serving
- Strong client-facing abilities
- Excellent collaboration and communication
- Mentoring and coaching mindset
- Databricks (Unity Catalog, DeltaLake)
- Python, PySpark
- Spark Structured Streaming
Reference: WJ-747_31182299