We are seeking a talented Data Engineer to join our dynamic team.
Bailey Abbott is a leading IT Consultancy business that brings amazing people together to explore the art of possible.
Job Overview
* This is an exciting opportunity for a skilled professional to contribute to our Data Intelligence capability. Our Data Intelligence specialists work within an active community of practice, enabling you to benefit from their years of collective experience and wisdom.
Data Engineering Role
* Represent us, sharing your skills and knowledge to help deliver projects for clients.
* Develop and maintain ETL (Extract, Transform, Load) pipelines to ingest, process, and transform data within our Microsoft Azure-based data lake, utilising Azure Data Factory.
* Design and implement efficient data models and schemas in Azure to ensure data quality and reliability, integrating Azure Purview for data governance.
* Utilise your proficiency in SQL and NoSQL databases to optimise data storage and retrieval in Azure environments.
* Leverage big data technologies such as Apache Spark, Hadoop, or Azure HDInsight to process and analyse large datasets.
* Collaborate with data scientists, analysts, and stakeholders to understand business requirements and deliver valuable data solutions tailored for Azure ecosystems.
* Write high-quality, scalable code in languages such as Python, Java, or Scala to support Azure-based data engineering tasks.
* Continuously monitor, troubleshoot, and optimise data pipelines in Azure for performance and reliability.
Requirements:
* Bachelor's degree in a related field or equivalent work experience.
* Demonstrated experience in data engineering or a related role, with a specific focus on Microsoft Azure.
* Proficiency in SQL and NoSQL databases, with a strong understanding of data manipulation and querying in Azure.
* Demonstrable experience in ETL pipeline development using Azure Data Factory, data modeling, and schema design.
* Familiarity with Azure big data technologies such as Apache Spark, Hadoop, or Azure HDInsight.
* Experience with Azure Purview for data governance and cataloging.
* Application of CI/CD and DevOps methodologies, including source code management.
* Programming skills in languages like Python, Java, or Scala, with a focus on Azure applications.
* Strong problem-solving skills and attention to detail.
* Effective communication and collaboration skills in an Agile team setting.
* Azure certification or relevant cloud experience is highly desirable.