Staff Data Engineer - Hybrid | 2 days on-site (Melbourne)
We're working with a highly regarded product-led technology organisation to hire a Staff Data Engineer into a mature, engineering-led data platform team operating at serious scale.
This is a hands‐on technical leadership role. You'll help shape architecture, guide engineering decisions and mentor others, while staying close to the code and delivery.
The platform processes millions of events per day and supports some of the organisation's most critical data workloads, so this role sits right at the centre of how the business operates.
What the role involves
You'll work across the design, build and operation of large‐scale data pipelines and platform components, with a particular focus on Spark‐based workloads.
Key areas include:
* Leading the delivery of Spark‐based data processing workloads within an AWS environment (primarily Databricks, with Spark on EMR also relevant)
* Designing and maintaining large‐scale batch and streaming data pipelines
* Owning performance, reliability and cost optimisation across high‐volume data workloads
* Working closely with engineering, analytics and business teams to deliver scalable, production‐ready data solutions
* Providing technical leadership and mentoring across the data engineering team
* Contributing to planning and delivery within an Agile environment with strong CI/CD and DevOps practices
What we're looking for:
We are specifically looking for engineers who are already operating at Staff / Lead Data Engineer level, with ownership of the architecture and evolution of large‐scale data platforms in production environments.
Your experience:
* Strong hands‐on experience with Apache Spark (Databricks strongly preferred)
* Python / PySpark expertise, with SQL used heavily in pipeline development and optimisation (exposure to Scala is a bonus)
* Experience building and supporting large‐scale data platforms and data warehousing environments on AWS
* Exposure to high‐volume batch and streaming data workloads
* Strong technical leadership capability, including mentoring engineers and guiding delivery decisions
* Ability to operate in complex environments where scale, reliability and performance really matter
This is a great opportunity for someone who enjoys owning complex data platforms, operating at scale, and being the engineer the team relies on when things matter most.
Australian Permanent Residency or Citizenship is required.
#J-18808-Ljbffr