Principal Data Engineer

As Principal Data Engineer, you shape the long-term data architecture for a real-time financial crime platform powering AML/KYC and fraud products. You will guide cross-tribe collaboration with engineering, data science, and product leads to tackle large-scale data problems and enable AI-driven detection. Your work centers on building and evolving a petabyte-scale data platform, knowledge graph pipelines, and robust data quality and observability. You will mentor engineers, represent the company externally, and drive data strategy that accelerates detection of financial crime at scale.

• Equity participation
• Unlimited time off
• Hybrid work (in-office two days/week)
• Learning budget
• Home office budget
• Enhanced parental leave and childcare benefits

• Define and own the medium-to-long term data platform architecture across multiple tribes
• Design and evolve pipelines for ingestion, transformation, and serving of large-scale signals
• Establish data quality, lineage, freshness, and observability standards for the knowledge graph
• Build and improve foundational data infra: ingestion frameworks, event bus, lakehouse, orchestration, developer experience
• Enforce consistent event-sourced and streaming patterns using Kafka and related tech
• Collaborate with ML/data science teams to support feature engineering, training pipelines, and online inference at scale
• Set data contracts, schema evolution standards, and API/event interface guidelines
• Mentor engineers, contribute to hiring and onboarding, and represent ComplyAdvantage at industry events

• Extensive experience designing and operating production-grade data platforms at large scale
• Deep expertise in distributed data systems (streaming, batch/ELT) and modern lakehouse/warehouse technologies
• Strong Python production experience; familiarity with Kotlin/Java for direction and mentoring
• Cloud experience (AWS/GCP) and containerized infra (Kubernetes, Docker, ArgoCD)
• Proven focus on data quality, observability, data contracts, and incident management for data systems
• Excellent written and verbal communication with ability to produce actionable technical docs
• Ownership mindset from inception through production and long-term operation
• Experience coaching engineers and influencing recruitment and onboarding

• Strong communication
• Collaborative leadership
• Mentoring and coaching
• Kafka and event streaming
• Spark, Flink, dbt, Airflow or Argo Workflows
• Lakehouse/warehouses technologies