Job Description
Data Engineer - Cloud Expertise
Halo Labs is a future-focused data solutions firm transforming tomorrow, today. Our intelligent and secure technology systems drive meaningful outcomes and unlock lasting value.
As a Data Engineer, you will design, develop, and enhance in-house PySpark frameworks to ensure standardised implementations and promote reusability across projects.
You will provide technical guidance and mentorship to junior team members, fostering skill development within the analytics community.
Collaborate with internal and external stakeholders to deliver high-quality, data-driven solutions that meet business needs.
Conduct exploratory data analysis and design end-to-end data engineering solutions using Azure Data Framework, Databricks, and other Azure services.
Build, test, and deploy scalable data pipelines using Python, PySpark, and Azure services.
Monitor and support batch operations for the Data and Insights Platform, ensuring reliability and performance.
Required Skills and Qualifications
* Cloud expertise with Azure Cloud Services, including Databricks, Azure Data Factory, Azure SQL Server, and Azure Storage.
* Strong experience with Databricks for big data processing.
* Proficiency in SQL, Python, and PySpark for data processing and pipeline development.
* Deep knowledge of data architecture, data modelling, data marts, and metadata management best practices.
* Minimum 3 years of experience working with modern data platforms (AWS, Azure) and data engineering.
* Expertise in Azure DevOps, including Git for version control and Azure Pipelines for CI/CD.
Benefits
Enjoy outstanding perks like a dedicated learning budget, performance bonuses, and comprehensive wellbeing support.
Experience the flexibility of remote work while having the opportunity to travel to client locations throughout Australia.
Others
A degree in Computer Science, Data Science, IT, or a related field is required.
While Queensland-based candidates are preferred, outstanding applicants from other locations will also be considered.