Overview
Job Description for Lead Data Engineer in Sydney
Responsibilities
* Leads data engineering initiatives with a focus on data warehousing, data analysis, visualization and data modeling.
* Designs and implements data solutions and pipelines using Python, Scala and SQL.
* Works with cloud platforms (Azure and Databricks) to build and maintain data infrastructure.
* Leverages Big Data technologies (Cloudera: HDFS, Spark, Hive, Kafka) and Apache Airflow for data workflows.
* Applies DevOps practices with Azure DevOps (Git, Repos, Pipelines), Git, Terraform and Docker.
* Supports reporting needs using SQL Server Reporting Services and Power BI.
Qualifications
* 10+ years of experience in software engineering, cloud computing, Databricks and Big Data.
* Data Engineering, Data Warehousing, Solution Design, Data Analysis, Visualization and Data Modeling.
* Programming: Python, Scala, SQL.
* Cloud: Azure and Databricks.
* Big Data: Cloudera (HDFS, Spark, Hive, Kafka), Apache Airflow.
* DevOps: Azure DevOps (Git, Repos, Pipelines), Git, Terraform, Docker.
* Reporting: SQL Server Reporting Services, Power BI.
* Certified DevOps Engineer Expert, Azure Cloud, Azure Data Lake.
#J-18808-Ljbffr