Lead Azure Databricks Data Engineer

Lead Azure Databricks data engineering with Python and Spark at Gore Mutual, building scalable infrastructure for enterprise data-driven decisions.

As Lead Data Engineer, you will design, develop, and maintain the Databricks lakehouse as the enterprise single source of truth. You will provide technical leadership while collaborating with data scientists, data product owners, IT, and business stakeholders. Your work supports policy, billing, claims, and finance reporting, pricing, and analytics.

Key Responsibilities:

• Build scalable data infrastructure and configuration-driven pipelines
• Implement pipeline data quality checks and security protocols
• Optimize pipeline flow, database performance, and platform integration
• Monitor platform health and resolve engineering issues
• Maintain backups, redundancy, disaster recovery, and current security patches

Requirements:

• Bachelor’s or Master’s degree in a related field
• Five to eight years of relevant data engineering experience
• Hands-on Azure Databricks and Microsoft Azure services knowledge
• Proficiency in Python, Spark, distributed systems, and SQL
• Experience with TDD, CI/CD, observability, and incident response

Apply your Azure Databricks, Python, Spark, and leadership expertise to Gore Mutual’s enterprise data platform.