Lead Data Engineer
Job Summary
Job Description
What is the opportunity?
As part of the P&CB Data & Analytics team you will generate value by transforming raw data from a complex array of source systems into user-friendly data assets which will be used by RBC's team of data scientists and insights analysts. You will develop, test, deploy, and maintain software to create complete and accurate data, using a variety of big data platforms, languages and tools, including Hadoop, Spark, Scala, Neo4j, Git and Maven. You will collaborate with a highly talented team of data engineers, data scientists, data architects, and data analysts in order to deliver business value to the organization. You will develop recommendations for new ways to improve data quality throughout the data transformation lifecycle.
What will you do?
Develop, test, and deploy software to generate data assets (relational, graph, other) for use by downstream insight analysts and data scientists
Work with big data, graph, and cloud technologies such as Spark, Hadoop, Hive, Neo4j, S3, EMR, EC2, Lambda, and Kafka
Work closely with stakeholders to ensure successful data asset design and development
Join data across multiple data environments, such as HDFS, S3 and Data Warehouses, using complex optimized queries
Use Scala, Cypher, Spark, GitHub, Maven, Jenkins and Airflow to develop and deploy automated data-producing software packages
Create software artifacts and patterns for reuse within the enterprise
Ensure ETL pipelines are produced with the highest quality standards, metadata and validated for completeness and accuracy
Work on a cross-functional Agile team responsible for end-to-end delivery of business needs
Help develop new solutions for batch and real-time data and analytics use cases
Help improve data management processes - acquiring, transforming and storing massive volumes of structured and unstructured data
Work closely with development teams to learn about needs, current processes and to promote best practices.
Must-have
Bachelors in Computer Science, Software Engineering, Mathematics, or other STEM majors. Masters/PhD considered an asset.
Professional software development experience with Python, Scala, Spark, Hadoop, Neo4j, Java, Linux, and SQL.
Experience with other analytics programming languages (Python)
3+ years of experience in the big data ecosystem, with Hadoop (Pig, Hive, HDFS), Apache Spark, and NoSQL/SQL databases
Experience using Git and Maven while collaborating on a software development team
Experience using ETL big data pipelines (Apache Airflow), knowledge of CI workflows and build/test automation
Nice-to-have
Experience with other data analytics and visualization tools such as Tableau
Experience with Agile software development
Experience with DevOps concerns, including CI/CD
What's in it for you?
We thrive on the challenge to be our best, progressive thinking to keep growing, and working together to deliver trusted advice to help our clients thrive and communities prosper. We care about each other, reaching our potential, making a difference to our communities, and achieving success that is mutual.
- A comprehensive Total Rewards Program including bonuses and flexible benefits, competitive compensation, commissions, and stock where applicable
- Leaders who support your development through coaching and managing opportunities
- Ability to make a difference and lasting impact
- Work in a dynamic, collaborative, progressive, and high-performing team
- A world-class training program in financial services.
- Opportunities to do challenging work, to take on progressively greater accountabilities and to building close relationships with clients
- Access to a variety of job opportunities across business and geographies.
#LI-Hybrid
#LI-Post
#LI-PK
Job Skills
Apache Airflow, Apache Hadoop, Apache Spark, Cloud Computing, Data Mining, ETL Processing, GitHub, Neo4j, Python (Programming Language), Quality Management, Requirements Analysis, Scala (Programming Language), Software Product Management, Structured Query Language (SQL), Waterfall ModelAdditional Job Details
Address:
City:
Country:
Work hours/week:
Employment Type:
Platform:
Job Type:
Pay Type:
Posted Date:
Application Deadline:
Note: Applications will be accepted until 11:59 PM on the day prior to the application deadline date above
Inclusion and Equal Opportunity Employment
At RBC, we embrace diversity and inclusion for innovation and growth. We are committed to building inclusive teams and an equitable workplace for our employees to bring their true selves to work. We are taking actions to tackle issues of inequity and systemic bias to support our diverse talent, clients and communities.
We also strive to provide an accessible candidate experience for our prospective employees with different abilities. Please let us know if you need any accommodations during the recruitment process.
Join our Talent Community
Stay in-the-know about great career opportunities at RBC. Sign up and get customized info on our latest jobs, career tips and Recruitment events that matter to you.
Expand your limits and create a new future together at RBC. Find out how we use our passion and drive to enhance the well-being of our clients and communities at jobs.rbc.com.