Senior Databricks Developer
Ayan Infotech is looking for a Senior Databricks Developer for a long-term contract job opportunity in Sydney.
Due to the nature of the work involved, we can only consider applicants who hold full unrestricted work rights in Australia and can work from the Sydney office two days a week at minimum.
Location: Sydney
Type: Contract (6 months +)
Required Skills and Experience
Minimum 5+ years of IT experience with 3+ years of hands-on experience in designing, developing, implementing, and maintaining data solutions with Databricks.
This role requires close collaboration with various teams to gather data requirements, build scalable data pipelines, and ensure the overall solution is reliable, available, and optimized for high performance.
Proven expertise with Lakehouse Architecture with Delta Tables – Delta tables, Schema Evolution, ACID compatibility, history capture, etc.
Integrating Databricks with various ETL/Orchestration tools and performing required operations.
Strong expertise in fundamental areas like Spark ecosystems, Data frame API & Spark SQL with Python/Scala and SQL as language.
Hands-on knowledge of Databricks native features like Workflows, Delta Live Tables, Unity Catalog, etc.
Expertise in workspace and cluster configuration, performance tuning, cost optimisation, data security and log monitoring.
Experience with Spark Structured Streaming jobs with Autoloader integrating with different streaming platforms.
Experience in building optimal data pipeline architecture with knowledge and best practices from Databricks platform.
Fair understanding of data modelling and defining conceptual, logical and physical data models.
Nice to have – exposure to AI/Gen AI skill and/or any associate or professional level Databricks/Cloud certifications.
We will be able to contact only shortlisted candidates.
We thank you in advance for your interest.
#J-*****-Ljbffr