Senior Data Engineer

BNA Senior Data Engineer: sits closer to the business outcome; specifically ensuring customer data is correctly selected & delivered for campaigns / customer communications.

CDI Senior Data Engineer / CRM Golden Layer Principal Data Engineer: owning the enterprise customer data model and golden layer within databricks. Blending engineering, modelling master data management, data architecture & technical leadership.

The organisation operates in the energy industry in London.

Description

• Own BNA pipeline architecture, orchestration & product readiness, implement arbitration, eligibility, suppression, consent, contact policy & cooldown logic. (Designing and maintaining pipelines, ensuring all jobs run in corrrect order and are ready for live use, implementing business rules).
• Own activation data sets / files, manifests, delivery status & SFTP integration - sent to marketing platforms / 3rd parties.
• Sequence & approve PRs (you are the technical authority before code reaches production).
• Enforce end-end regression and production validation
• Own production cutover, run/recovery, observability, alerting and incident technical response.
• Own CRM gold and associated golden layer facts / dimensions (accountable for facts like website activity)
• Design customer, identifier, consent translation, redemption and feedback models (define how customer info is structured).
• Manage source to target mapping, SCD/history handling, lineage & DQ (understanding how customer data moves from source to gold layer)
• Own databricks workflow scheduling, trigger dependencies, freshness & runtime optimisation
• Resolve cross-system data quality & historical reload issuesDefine technical standards, documentation and interface contracts for consumers

Profile

Advanced SQL, Python/Spark and Databricks data engineering.

Workflow orchestration, dependency management, idempotency and scheduled/triggered jobs.

CI/CD, version control Azure DevOps, PR review, unit/integration testing and environment promotion. Production support, DQ controls, alerting, reconciliation and incident diagnosis

Nice to have: CRM decisioning, history with customer eligibility/consent and activation; Practical Comarch and/or SFMC/SFTP interface.

Expert SQL, Python/Spark, Databricks and dimensional/data-product modelling.

SCD1/SCD2, CDC, historical backfill and high-volume transaction processing.

Orchestration, CI/CD, testing, observability and production support.

Nice to have: CRM, loyalty, consent and customer-identity domain work. Ability to work with CIS, Comarch, SFMC, AWS/Data Hub and analytics consumers.

Job Offer

Based in Canary Wharf, Competitive daily rate of £600-£800 a day inside IR35. 12 months contract.