Randstad is currently working with the City of Moreton Bay to find a Principal Data Engineer (Microsoft Fabric). This role is based in Moreton Bay with WFH flexibility (3 days per week WFH).
We are seeking a Principal Data Engineer (Microsoft Fabric) to serve as the ultimate technical authority and domain expert. In this pivotal role, you will lead the end‐to‐end architecture, governance, and operational excellence of our Microsoft Fabric platform, establishing a unified and trusted data foundation that enables next‐generation Copilot integrations, real‐time analytics, and advanced AI/ML capabilities.
What You Will Do:
* Serve as the technical authority for all Microsoft Fabric components (OneLake, Lakehouse, Data Warehouse, and Real‐Time Analytics) and establish enterprise‐grade governance using Microsoft Purview.
* Own the Fabric platform's operational model, including capacity planning, workspace topology design, environment lifecycle management, and rigorous cost optimization.
* Lead the design and development of modular data products, Lakehouse schemas, and scalable ETL/ELT pipelines using Data Factory and Spark Notebooks for high‐volume workloads.
* Own the DataOps/CI‐CD strategy and operational monitoring, leveraging Git integration and Deployment Pipelines to automate promotion across environments and maintain strict data quality SLAs.
* Set the gold standard for engineering excellence, mentoring data engineers and collaborating with data scientists to design data processing components for MLOps workflows.
What You Need to Apply for This Role:
* Mandatory, deep hands‐on expertise across the entire Microsoft Fabric ecosystem alongside expert‐level proficiency in Microsoft Purview for data governance and lineage.
* Expert‐level coding proficiency in Python (for Spark/Notebooks) and SQL (T‐SQL/Spark SQL), with strong fundamentals in distributed systems and data modelling.
* Proven experience implementing DataOps, Infrastructure‐as‐Code (IaC), and automated CI/CD pipelines using GitHub, Azure DevOps, or Fabric Deployment Pipelines.
* Demonstrated ability to lead with influence, drive architectural consensus, and manage multi‐disciplinary stakeholders across engineering, data science, and product teams.
Benefits:
* Access to a Udemy license for ongoing learning and continuous professional development.
* Access to exclusive discounts across various platforms.
* Discounts on private health funds.
* Three payruns a week.
At Randstad Digital, we are passionate about providing equal employment opportunities and embracing diversity to the benefit of all. We actively encourage applications from any background.
#J-18808-Ljbffr