As a Senior Consultant - Data Engineering, you will play a pivotal role in modernizing the technological infrastructure by implementing comprehensive analytics and data platforms using Azure and Microsoft Fabric. This is a highly hands‐on role, where you will be directly involved in the design, development, and management of data pipelines, ETL processes, and integration with critical business applications. You will work closely with cross‐functional teams to deliver scalable, efficient, and innovative data solutions.
Key Responsibilities
* Lead the implementation of an end-to-end analytics and data platform (Databricks/Microsoft Fabric).
* Develop, maintain, and optimize ETL processes to ingest and process data from various sources, including SQL Server, Salesforce, and other systems, into Azure Fabric's OneLake or Databricks.
* Create and refine data models, data pipelines, and data processing workflows, ensuring efficiency, scalability, and performance.
* Implement and manage CI/CD pipelines for data engineering tasks to enhance deployment efficiency, reliability, and maintainability.
* Support, maintain, and optimize data warehouse and data lake solutions, ensuring they meet business requirements and performance standards.
* Provide database administration services, focusing on data integrity, performance tuning, governance, quality, and security.
* Work closely with data scientists and analysts to automate the distribution of insights and predictions through Power BI and other tools/platforms.
* Act as a technical mentor to junior team members, fostering a collaborative, innovative, and learning-focused environment.
* Effectively communicate with various business units to understand their data needs and deliver appropriate, tailored solutions.
* Stay updated with emerging technologies, particularly in AI/ML, and integrate innovative solutions into the data architecture.
Desired Skills And Experience
* Proven hands‐on experience in ETL development, report development, database design, and administration, with a strong focus on Azure and Microsoft Fabric or Databricks.
* Strong expertise in data modeling, data pipeline development, data integration, database management, and data processing optimization.
* Experience with Azure Data Services such as Azure Data Factory, Azure Synapse Analytics, Azure Data Lake Storage.
* Proficiency in programming languages such as Python.
* Working experience with CI/CD tools and practices, particularly Azure DevOps, for data engineering tasks.
* Proficient in implementing data governance frameworks and tools such as Microsoft Purview.
* Understanding of machine learning concepts and experience working with data scientists to operationalize ML models.
* Excellent communication skills to interact with various business units and understand their data needs.
Preferred Qualifications
* Advanced degree in Computer Science, Data Science, Information Technology, or a related field.
* Certifications in Azure or Databricks.
* Familiarity with other cloud platforms and tools such as AWS, GCP.
#J-18808-Ljbffr