This is a senior data engineer position at a top ASX-listed company. We are seeking an experienced professional to join our team and contribute to the design, development, and maintenance of scalable AWS Glue pipelines.
The ideal candidate will have a strong background in data engineering with experience working on complex datasets and cloud-based pipeline development. They will be responsible for designing, building, and optimizing scalable AWS Glue (PySpark) pipelines for structured and semi-structured data sources.
Key Responsibilities:
We are looking for someone who can deliver clear and accurate source-to-target mappings and integration specifications across complex datasets. The successful candidate will also be responsible for ingesting and transforming data from multiple systems into S3, Elastic, and downstream platforms.
Apart from technical skills, the candidate should be able to collaborate with cross-functional teams including architects, business SMEs, and ETL/BI developers. They will ensure that pipelines are production-grade, fault-tolerant, and aligned to data governance and security standards.
Required Skills & Qualifications:
To succeed in this role, the candidate must have a minimum of 5 years of Data Engineering experience in enterprise environments. They should possess strong hands-on expertise in AWS Glue (PySpark) and AWS native services such as S3, Lambda, Athena, and Step Functions.
Practical experience integrating with Elastic/Elasticsearch for data indexing and retrieval is also essential. The candidate should have a deep understanding of ETL/ELT design patterns, CDC, and data modeling.
Solid grasp of source-to-target mapping principles and best practices is required. Additionally, they should have strong SQL and Python skills.
Benefits:
This is a mid-senior level contract position with flexible working arrangements. The company offers a hybrid model with a mix of office and remote work to support work-life balance.
We value diversity and encourage applications from qualified candidates. To apply, please submit your resume and cover letter outlining your experience and qualifications.