Join to apply for the Staff / Lead Data Engineer (AWS, Big Data) role at Commonwealth Bank.
About the role
As a Staff Data Engineer with expertise in software development and a passion for building data‑driven solutions, you’ll work at the forefront of Big Data and Data Warehouse technologies. Your work will help improve the standard of digital banking, using the latest tech to solve our customers’ most complex data‑centric problems.
The Risk Data Analytics and Insights team delivers data, insights and software outcomes to support Risk Management and other business units. You’ll work across platforms such as AWS Cloud, Cloudera Hadoop, Teradata, Ab Initio, and emerging Generative AI solutions.
Qualities we look for
- Experience driving cost‑effective, technologically feasible solutions that meet operational and strategic goals.
- Thought leadership and contribution to design of technical solutions, including data security and data management.
- Responsibility for technical governance, implementation, risk management and adoption for programs of work.
- Technical leadership and mentoring of engineers in Agile teams delivering project work or initiatives.
- Solid communication and coordination skills with internal stakeholders and consultants.
- Ability to build group data products from scratch, integrating large data sets from internal and external sources in alignment with data mesh principles, patterns, guardrails or architecture blueprints.
Technical Skills
- Designing, building, and delivering enterprise‑wide data ingestion, integration and pipeline solutions using Scala, Java or Python on Big Data and Data Warehouse platforms.
- Experience with Hadoop, Spark, MapReduce, Sqoop, Kafka and ETL frameworks for distributed data processing and storage.
- Experience with AWS Cloud technologies such as EMR, Glue, Kinesis, MSK/Kafka, Data Zone, SageMaker, Event Bridge, MWAA, Redshift/PostgreSQL, DocumentDB/MongoDB, S3, Athena, Lambda.
- Proficiency in SQL and complex query writing for pipeline construction.
- Experience with source control, DevSecOps, CI/CD tooling (GitHub Actions), CloudFormation, GitOps and serverless frameworks.
- Ability to fine‑tune ETL jobs, query performance and cost efficiencies.
- Knowledge of data warehousing and/or data mart build experience in Teradata is a plus.
- Certifications in Cloudera CDP, Hadoop, Spark, Teradata or AWS Data Engineer are desirable.
Working with us
We value diversity, inclusion and flexibility. You can work remotely most days but will connect with us at least half a month’s time in the office. Flexible schedule, part‑time arrangements and job‑share options are also available.
If this sounds like the role for you, we would love to hear from you. Apply today!
For Commonwealth Bank Group members, please apply through Sidekick to submit a valid application.
#J-18808-Ljbffr