Big Data Developer
Centraprise
Position: Big Data Developer
Location: Toronto, ON (5 Days Onsite)
Duration: Contract-Long term
Mandatory Skills: Well versed with Data, ETL Processes, SQL and high-level programming languages such Scala, Java, DevSecOps
We're Hiring: Big Data Developer
We’re looking for a Big Data Developer to join an Agile team and build scalable data pipelines, ETL processes and modern data solutions.
What You'll Do
- Design, develop and maintain scalable data ingestion pipelines
- Build and optimize ETL processes and data pipelines
- Work with Snowflake and integrate it with various data sources and BI platforms
- Develop and optimize complex SQL queries, stored procedures and views
- Work with Big Data technologies including Spark, Hadoop and Hive
- Develop solutions using Scala or Java
- Monitor, troubleshoot and optimize data jobs and platform issues
- Ensure data quality, security, reliability and performance
- Create and maintain documentation for data pipelines, transformations and data models
- Collaborate with developers, data engineers and business stakeholders in an Agile environment
- Contribute to CI/CD, DevSecOps and cloud-based data engineering practices
- Leverage GenAI tools to improve development productivity and code generation
What We're Looking For
Required Skills
- 5+ years of experience with Big Data technologies used for ETL
- Hands-on experience with Snowflake
- Strong SQL and data modeling skills
- Experience with Spark, Hadoop and/or Hive
- Programming experience with Scala or Java
- Experience developing or supporting data pipelines / ETL processes
- Experience with ETL tools such as Informatica, Talend or Apache Airflow
- Experience with at least one cloud platform, preferably AWS
- Understanding of SCM, CI/CD and DevSecOps
- Strong analytical, troubleshooting and communication skills
Nice to Have
- Apache Airflow / workflow orchestration
- Git / GitHub / GitHub Actions
- Automated testing
- Docker / Kubernetes / Containers / OCP4
- Shell scripting
- Infrastructure-as-Code
- Python or other scripting languages
- GenAI / AI-assisted development experience
Reference: WJ-3844_2752615