Unlock a world of data-driven opportunities as a Senior Data Engineer. Work at the forefront of Cloud, Big Data and Data warehouse technologies, staying ahead of the latest trends and innovations.
Join a dynamic team of engineers who are passionate about building state-of-the-art data solutions that power seamless experiences for millions of customers.
As a Senior Data engineer with expertise in software development / programming and a passion for building data-driven solutions, you'll be part of a collaborative environment where ideas are shared, and initiatives are encouraged.
To us, data is everything - it powers our cutting-edge features and enables us to provide exceptional customer experiences.
We're responsible for CommBank's key analytics capabilities and work to create world-leading capabilities for analytics, information management, decisioning and Generative AI.
Key Responsibilities
* Design, build and deliver enterprise-wide data ingestion, data integration and data pipeline solutions using common programming languages (Scala, Java or Python).
* Build data solutions in Hadoop platforms, utilizing Spark, MapReduce, Sqoop, Kafka and various ETL frameworks for distributed data storage and processing.
* Experience in building data solutions using AWS Cloud technology (EMR, Glue, Iceberg, Kinesis, MSK/Kafka, Redshift/PostgresSQL, DocumentDB/MongoDB, S3, etc.).
* Possess ability to produce conceptual, logical and physical data models using data modelling techniques such as Data Vault, Kimball, 3NF, etc. and demonstrate expertise in design patterns (FSLDM, IBM IFW DW).
* Strong Unix/Linux Shell scripting and programming skills in Scala, Java, or Python.
* Proficient in SQL scripting, writing complex SQLs for building data pipelines.
* Familiarity with data warehousing and/or data mart build experience in Teradata, Oracle or RDBMS system is a plus.
* Experience in Ab Initio software products (GDE, Co Operating System, Express It, etc.) is a plus.
Requirements
* Confident in building group data products or data assets from scratch, by integrating large sets of data derived from hundreds of internal and external sources.
* Can collaborate, co-create and contribute to existing Data Engineering practices in the team.
* Can lead and mentor other engineers in Agile teams delivering project work or initiative.
* Have experience and responsible for data design, data security and data management.
* Have a natural drive to educate, communicate and coordinate with different internal stakeholders and consultants.
Benefits
Our people bring their diverse backgrounds and unique perspectives to build a respectful, inclusive, and flexible workplace with flexible work locations.
One where we're driven by our values, and supported to share ideas, initiatives, and energy.
Here, you'll thrive. You'll be supported when faced with challenges and empowered to tackle new opportunities.
About Us
We use a broad range of tools, languages, and frameworks.
We don't expect you to know them all but experience or exposure with some of these (or equivalents) will set you up for success in this team.