Principal Data Engineer
We are seeking an exceptional data professional to join our team as a Principal Data Engineer. This role will involve leading the design, development and maintenance of end-to-end data analytical solutions using Microsoft on-premises and cloud technologies.
Responsibilities:
* Lead the design, development, and maintenance of end-to-end data analytical solutions using Microsoft on-premises and cloud technologies.
* Engage in collaborative requirement elicitation activities with clients.
* Mentor and guide junior team members in the field of data engineering.
* Deliver innovative analytical solutions to clients and contribute to knowledge transfer within the organization.
Qualifications:
* 5+ years of experience as a data engineer.
* Familiarity with analytical architectures including Data Warehouses, Data Lakes and Data Lakehouses.
* Knowledge of Microsoft relational engines available – both on-premises (MS SQL Server) and on the cloud (Azure SQL, Azure Synapse Analytics Dedicated Pools).
* Knowledge of query and workload optimization.
* Knowledge of administrating and managing analytical solutions both on premises and on the cloud.
* Knowledge of NoSQL offerings on Azure (e.g. CosmosDB).
* Knowledge of setting up Data Lakes on ADLS gen 2 and Azure Synapse Analytics.
* Knowledge of using SSIS and Azure Data Factory to set up ETL jobs.
* Knowledge of setting up OLAP solutions using SSAS.
* Familiarity with Azure Data Catalog to manage data lake metadata.
* Knowledge of governance over Azure based analytical solutions including using tools such as Purview, Active Directory, Key Vault, Azure DevOps, etc.
* Knowledge of using Azure Stream Analytics to capture and analyze data in real time.
* Knowledge of setting up data warehouse solutions both on-premises (using MS SQL Server) and on the cloud (using Azure Synapse Analytics).
* Knowledge of Dimensional Modeling (Dimensions, Facts, Slowly Changing Dimensions, Outriggers, Role Playing Dimensions, Junk Dimensions, Degenerate Dimensions, Multi-valued Dimensions, Transactional Facts, Periodic Snapshot Facts and Accumulating Snapshot Facts).
* Knowledge of implementing common ETL patterns to support data warehouse workloads.
* Knowledge of using U-SQL, PolyBase and Azure Data Lake Analytics to perform federated querying.
* Knowledge of PowerBI.
* Familiarity with SSRS.
* Snowflake Experience: Experience with Snowflake is required.
* Nice to Have: MSc in Computer Science or other relevant field.
* Familiarity with Azure Databricks.
* Familiarity with DataOps.
* Familiarity with Azure Cognitive Services.
* Familiarity with Azure ML.
* Familiarity with MLOps.
* Familiarity of containerization tools and frameworks (Docker, Kubernetes, etc.).
* Familiarity with Azure HD Insight.
* Familiarity with the Hadoop ecosystems, including tools like Hive, Spark and Kafka.
* Familiarity with Azure Functions to perform lightweight transformations.
Benefits:
* Salary paid in USD.
* 6-month career advancing opportunities.
* Supportive and friendly work environment.
* Premium medical insurance (employee + family).
* English language development courses.
* Interest-free loans paid over 2.5 years.
* Technical development courses.
* Planned overtime program (POP).
* Employment referral program.
* Premium location in Maadi.
* Social insurance.