Data Engineer for On-Chain Pipelines
CLR3
Join datastore in building reliable data pipelines that decode years of on-chain history into trustworthy Parquet files for researchers. Elevate data correctness and infrastructure while ensuring high quality and accuracy.
As a Data Engineer, you will design and run large-scale decoding pipelines, focusing on Solana and Hyperliquid history. This role demands significant experience in building production data pipelines, emphasizing SQL alongside either Python, Rust, or Go. You'll ensure correctness through rigorous validation, checksums, and lineage documentation, addressing customer inquiries about schemas and coverage.
Key Responsibilities: • Design and implement backfill pipelines at scale • Model typed schemas for various protocols • Build validation systems for data correctness • Maintain accurate versioning and documentation • Optimize storage layout for fast query performance
Requirements: • Proven experience in production data pipeline development • Proficiency in SQL plus Python, Rust, or Go • Familiarity with Parquet and tools like DuckDB • Strong focus on data correctness and validation • Willingness to work part-time in Toronto office
Help create a reliable data pipeline that researchers can trust, ensuring high-quality deliveries every time. #J-18808-Ljbffr
As a Data Engineer, you will design and run large-scale decoding pipelines, focusing on Solana and Hyperliquid history. This role demands significant experience in building production data pipelines, emphasizing SQL alongside either Python, Rust, or Go. You'll ensure correctness through rigorous validation, checksums, and lineage documentation, addressing customer inquiries about schemas and coverage.
Key Responsibilities: • Design and implement backfill pipelines at scale • Model typed schemas for various protocols • Build validation systems for data correctness • Maintain accurate versioning and documentation • Optimize storage layout for fast query performance
Requirements: • Proven experience in production data pipeline development • Proficiency in SQL plus Python, Rust, or Go • Familiarity with Parquet and tools like DuckDB • Strong focus on data correctness and validation • Willingness to work part-time in Toronto office
Help create a reliable data pipeline that researchers can trust, ensuring high-quality deliveries every time. #J-18808-Ljbffr
Reference: WJ-3875_13619500