Streaming Engineer – Payments | Kafka | Pyspark
Location: Sydney - hybrid
Contract: 6‐Month Contract
Start: ASAP
About our client
Our client is a leader in professional services and consulting, providing a wide range of services including strategy, consulting, technology and systems integration, and operations to a range of clients across Government, Financial Services, Utilities and more.
About the Role
We're seeking an experienced Streaming Engineer to join a high‐performing data engineering team delivering near real‐time data solutions with a banking client. You'll design, build and support modern streaming pipelines using PySpark, Kafka and DynamoDB, with a strong emphasis on cloud (AWS preferred), performance, scalability, and reliability.
Key Responsibilities
* Develop, test and support batch and real‐time data pipelines.
* Build future‐ready data solutions across multiple industry verticals.
* Implement distributed streaming applications using PySpark, Kafka & Dynamo.
* Design and deliver ETL, data warehousing and BI solutions.
* Create Proof of Concepts and working demos for stakeholders.
* Collaborate with consultants, architects and technical teams across engagements.
* Mentor junior team members and contribute to best‐practice delivery.
Mandatory Experience
* 10+ years in solution design, development, testing or support across IT consulting or technology environments.
* 3+ years hands‐on with Big Data/Data Lake platforms (cloud + on‐prem).
* Proven experience delivering real‐time streaming pipelines.
* Strong skills in Python, Scala, SQL, shell scripting.
* Experience working with AWS, Azure, GCP or IBM cloud platforms.
* Deep understanding of containerisation, virtualisation and networking.
* Hands‐on experience building solutions meeting high availability, fault tolerance, auto failover and performance SLA requirements.
* Background in Agile, Kanban or Waterfall delivery frameworks.
* Strong stakeholder communication and leadership capability.
Why Join?
* Work on enterprise‐scale data and streaming projects.
* Join a collaborative, high‐calibre engineering team.
* Opportunity to influence architecture and technical strategy.
* Hybrid work environment and excellent team culture.
#J-18808-Ljbffr