AI/ml Data Engineer

What is the opportunity?

As a Data Engineer :

• Creating software for retrieving, parsing and processing structured and unstructured data.
• Defining and building scalable ETL/ELT workflows for reporting and analytics.
• Supporting project team to scale, monitor and operate data platforms for very high availability and performance.
• Developing scripts and programs for converting various types of data into usable formats.

What will you do?

• Develop, test, and deploy software to generate data assets (relational, graph, other) for use by downstream insight analysts and data scientists
• Work with big data, graph, and cloud technologies such as Spark, Hadoop, Hive, Neo4j, S3, EMR, EC2, Lambda, and Kafka. Work closely with stakeholders to ensure successful data asset design and development
• Join data across multiple data environments, such as HDFS, S3 and Data Warehouses, using complex optimized queries
• Use Scala, Cypher, Spark, GitHub, Maven, Jenkins and Airflow to develop and deploy automated data-producing software packages
• Create software artifacts and patterns for reuse within the enterprise
• Ensure ETL pipelines are produced with the highest quality standards, metadata and validated for completeness and accuracy
• Work on a cross-functional Agile team responsible for end-to-end delivery of business needs
• Help develop new solutions for batch and real-time data and analytics use cases. Help improve data management processes - acquiring, transforming and storing massive volumes of structured and unstructured data
• Work closely with development teams to learn about needs, current processes and to promote best practices.

What do you need to succeed?

Must-have

• Bachelors in Computer Science or Software Engineering
• Min 3 years of experience with Python, Spark, Hadoop, Java, Linux, and SQL
• Experience in the big data ecosystem, with Hadoop (Pig, Hive, HDFS), Apache Spark, and NoSQL/SQL databases
• Experience using Git or Maven while collaborating on a software development team
• Experience using ETL big data pipelines, knowledge of CI workflows and build/test automation

Nice-to-have

• Experience with other analytics programming languages (R)
• Experience with other data analytics and visualization tools such as Tableau
• Experience with Agile software development
• Experience with DevOps concerns, including CI/CD

What's in it for you?

We thrive on the challenge to be our best, progressive thinking to keep growing, and working together to deliver trusted advice to help our clients thrive and communities prosper. We care about each other, reaching our potential, making a difference to our communities, and achieving success that is mutual.

• A comprehensive Total Rewards Program including bonuses and flexible benefits, competitive compensation, commissions, and stock where applicable
• Leaders who support your development through coaching and managing opportunities
• Ability to make a difference and lasting impact
• Work in a dynamic, collaborative, progressive, and high-performing team
• A world-class training program in financial services
• Opportunities to do challenging work, to take on progressively greater accountabilities and to building close relationships with clients
• Access to a variety of job opportunities across business and geographies.

#LI-Post

#LI-Hybrid

#LI-PK

#TECHPJ

Job Skills

Agile Methodology, Agile Methodology, Apache Hadoop, Apache Spark, Big Data, Big Data Management, Cloud Computing, Cloud Computing Architecture, Data Analytics, Data Architecture, Database Development, Database Queries, Data Governance, Data Mining, Data Pipelines, Data Warehousing (DW), Design, DevOps, End-to-End Testing, ETL Pipelines, ETL Processing, Git, Group Problem Solving, Python (Programming Language), Quality Management {+ 2 more}

Additional Job Details

BAY WELLINGTON TOWER, 181 BAY ST:TORONTOTORONTOCanada37.5Full timeSTRATEGY AND ADMINISTRATIONRegularSalaried2024-10-022024-10-31