Senior Data Engineer
London Stock Exchange
- We are seeking a highly skilled and innovative Senior Data Engineer to join our team
- In this role, you will be responsible for designing, building, and managing our data infrastructure and analytics solutions to enable data-driven decision-making across the organization
- You will work on complex datasets and collaborate with cross-functional teams to unlock insights and deliver actionable intelligence
- Design, develop, and optimize big data pipelines, architectures, and data integration workflows
- Build scalable and efficient data storage solutions, such as data lakes, warehouses, or cloud-based systems
- Collaborate with data scientists, analysts, and stakeholders to understand data requirements and deliver solutions
- Develop and maintain ETL processes for ingesting, transforming, and loading data from diverse sources
- Conduct advanced data analysis, modelling, and visualization to identify patterns and insights
- Ensure data quality, reliability, and security across systems
- Work with distributed computing frameworks (e.g., Spark, Hive, Hadoop) to process and analyse large datasets
- Stay up to date with the latest trends and technologies in big data, analytics, and cloud platforms
Hands-on experience with Databricks, SQL, and data modellingExcellent problem-solving and interpersonal skillsSolid grasp of ETL workflows within databases / warehouses such as Oracle, PostgreSQL, SQL Server etcProficiency with big data tools (e.g., Apache Spark, Apache Kafka, Presto, etc.) and cloud services (e.g., Azure, AWS, GCP)Bachelor’s / master’s degree or equivalent experience in computer science engineering, or a related field10+ years of meaningful experience in big data and analyticsStrong programming skills in languages such as Python, Java, or ScalaFamiliarity with data visualization tools and business intelligence (e.g., Power BI)Experience with machine learning frameworks and tools such as Azure ML Studio, AWS Sagemaker etcKnowledge of real-time data processing and stream analytics such as Apache Kafka, Azure Event Hubs or similarHands on experience with real-time analytical databases such as Clickhouse or Apache Druid/Pinot or similar
#J-18808-LjbffrReference: WJ-766_21913661