Borg Group is embarking company-wide shift to a modern Lakehouse platform on Microsoft Fabric, and we're looking for a skilled Data Engineer to help build and scale our modern data platform. This role is based onsite 5 days a week at our Somersby Head Office, offering the opportunity to work closely with stakeholders.
The Role
As a Data Engineer, you will design and build scalable data pipelines, support a robust Medallion architecture, and help drive automation, governance, and performance across the platform.
Key Responsibilities
* Design and implement scalable data ingestion pipelines using the Metadata Framework (MDF), initially onboarding Dynamics and API sources, with additional systems to follow
* Develop and maintain PySpark and Spark SQL notebooks across Bronze, Silver, and Gold layers, supporting data cleansing, schema drift handling, and incremental processing
* Configure and manage MDF metadata JSON files to orchestrate logic across batch, staging, and processing layers, enabling rapid onboarding of new data sources
* Support CI/CD pipeline development using Azure DevOps and Terraform (IaC), enabling consistent and scalable deployment of Fabric components across environments
* Implement and maintain data security standards including Row Level Security (RLS), RBAC via Entra ID, and column-level security to support scalable governance
* Monitor and optimise platform performance using Log Analytics, Fabric Monitoring Hub, and Real-Time Hub alerting to ensure SLA compliance as data volumes grow
Who You Are
* Experienced in building and optimising data pipelines using Spark, PySpark, or similar big data technologies
* Strong understanding of modern data architectures, including Lakehouse concepts and medallion (Bronze/Silver/Gold) frameworks
* Hands-on experience with Azure data services (e.g., Azure DevOps, Fabric, Data Lake, or similar cloud platforms)
* Comfortable working with infrastructure-as-code tools such as Terraform and implementing CI/CD practices
* Knowledge of data governance, security models, and best practices in access control and compliance
* A proactive problem-solver who thrives in a collaborative, onsite team environment and enjoys building scalable, future-focused data solutions
Why Join Borg Group?
* Be part of a major data transformation initiative with real business impact
* Work with modern technologies in a growing data function
* Collaborate closely with stakeholders in an onsite, team-oriented environment
* Opportunity to shape and influence the future state of data at Borg