We are seeking a skilled and proactive Data Engineer with hands-on experience in Python, PySpark, and AWS Glue or DynamoDB for one of our clients based in Sydney. The ideal candidate will be responsible for building, optimizing, and managing data pipelines and workflows for a scalable cloud-based data platform.
Key Responsibilities:
* Design and develop ETL/ELT data pipelines using PySpark and Python.
* Work with AWS Glue or DynamoDB to ingest, process, and manage structured and semi-structured data.
* Collaborate with data analysts, scientists, and engineering teams to deliver reliable, high-performance data solutions.
* Implement data quality checks, validation rules, and logging mechanisms.
* Monitor pipeline performance, troubleshoot failures, and ensure SLAs are met.
* Document pipeline processes and provide technical knowledge sharing across teams.
Required Skills & Experience:
* 3+ years of experience in a data engineering or related role.
* Strong proficiency in Python for data transformation and scripting.
* Solid hands-on experience with PySpark for big data processing.
* Experience with AWS Glue or DynamoDB for cloud-native data handling.
* Understanding of data formats such as JSON, Parquet, CSV, and Avro.
* Familiarity with AWS services like S3, Lambda, IAM, CloudWatch, or Athena.
* Experience with version control (e.g., Git) and CI/CD pipelines.
Seniority level
* Seniority level
Mid-Senior level
Employment type
* Employment type
Full-time
Job function
* Job function
Information Technology
* Industries
IT Services and IT Consulting
Referrals increase your chances of interviewing at Kaizen Global Technologies by 2x
Sign in to set job alerts for "Data Engineer" roles.
Macquarie Park, New South Wales, Australia 4 months ago
Sydney, New South Wales, Australia 3 days ago
Surry Hills, New South Wales, Australia 3 weeks ago
Minchinbury, New South Wales, Australia 3 days ago
Pyrmont, New South Wales, Australia 1 month ago
Sydney, New South Wales, Australia 18 hours ago
Sydney, New South Wales, Australia 5 days ago
Sydney, New South Wales, Australia 2 weeks ago
Baulkham Hills, New South Wales, Australia 3 weeks ago
Sydney, New South Wales, Australia 5 days ago
Software Engineer, Early Career, NetSoft
Sydney, New South Wales, Australia 1 day ago
Macquarie Park, New South Wales, Australia 7 months ago
Sydney, New South Wales, Australia 1 month ago
Sydney, New South Wales, Australia 6 days ago
Sydney, New South Wales, Australia 3 weeks ago
Sydney, New South Wales, Australia 3 weeks ago
Greater Sydney Area A$140,000.00-A$190,000.00 2 weeks ago
Kensington, New South Wales, Australia 3 weeks ago
Sydney, New South Wales, Australia 3 weeks ago
Data Engineer - Brisbane, Sydney or Perth location
Sydney, New South Wales, Australia 4 weeks ago
Sydney, New South Wales, Australia 2 days ago
Sydney, New South Wales, Australia 3 weeks ago
Software Engineer (Python) - Data Platform, remote across ANZ
Sydney, New South Wales, Australia 5 days ago
Sydney, New South Wales, Australia 1 day ago
Sydney, New South Wales, Australia 6 days ago
Sydney, New South Wales, Australia 3 weeks ago
Sydney, New South Wales, Australia 22 hours ago
We're unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.
#J-18808-Ljbffr