Data Engineering Role
We are seeking a skilled Data Engineer to join our team. As a Data Engineer, you will play a crucial role in architecting, designing, developing, testing, and implementing scalable, robust, and efficient real-time data pipelines using Kafka.
Key responsibilities include:
* Working closely with key stakeholders to comprehend and fulfill complex data integration needs.
* Architecting, designing, developing, testing, and implementing scalable, robust, and efficient real-time data pipelines using Kafka.
* Developing and managing ETL/ELT processes and workflows using Python to ensure optimal data quality and efficiency.
* Crafting and executing SQL queries, stored procedures, and views within Snowflake data lake/warehouse.
* Driving the design, development, and maintenance of our AWS architecture.
* Fostering a data-driven culture and advocating for robust, scalable, and reliable data infrastructure.
* Developing and maintaining Infrastructure-as-Code (IAC) for the Integration Platform as a Service (IPaaS).
* Implementing and utilising CI/CD pipelines to automate testing and deployment.
* Developing IRT's integration and system monitoring capabilities, automatically providing detailed alerting to integration issues, abnormal billing, and system downtime.
In order to be successful, you will have:
* Bachelor's Degree in Computer Science, Mathematics or equivalent.
* 5+ years' experience in a data-centric role.
* Proven experience in using Kafka, Terraform, Buildkite, and AWS or similar services.
* Strong Python coding skills.
* Experience in building and maintaining data pipelines.
* Understanding of data architecture and infrastructure-as-code (IaC) principles.
* Ability to prioritize, multitask and work to deadlines.
* Excellent attention to detail and accuracy.
* Proven problem-solving skills.
* DevOps experience (GitHub, CI/CD).
* Demonstrated willingness to work as part of a multidisciplinary team.
We offer competitive pay and more cash in your pocket (less tax) with not-for-profit salary packaging, flexible working conditions, birthday leave, professional and career development opportunities, multiple career pathways, discounted gym memberships, free counselling via Employee Assistance Program (EAP) and staff wellness program. If you feel this is the right role for you, we'd love to hear from you. Simply click the 'Apply now' button and submit your details. Once you apply, we'll be in touch to discuss your application.