Ayan Infotech is seeking a highly skilled Microsoft Fabric Engineer to design and implement real-time data ingestion and processing pipelines using Microsoft Fabric Real-Time Analytics (RTA) or Azure Data Explorer (ADX). The ideal candidate will have hands-on experience with Azure Data Explorer, expertise in architecting, designing, and optimizing real-time data pipelines, as well as strong knowledge of Kusto Query Language (KQL) and ADX data modeling.
Key Responsibilities:
* Design and implement real-time data ingestion and processing pipelines
* Develop scalable solutions for processing high-volume time-series data from IoT devices, sensors, and telemetry sources
* Optimize data models for efficient storage, querying, and analytics in Fabric or ADX
Data Engineering & Integration:
* Work with KQL to build efficient queries for time-series analysis
* Develop connectors and integrations with event streams (IoT Hub, Event Hubs, Kafka, etc.) for real-time data ingestion
* Implement data transformation, enrichment, and aggregation strategies for downstream analytics and dashboards
Performance Optimization & Reliability:
* Tune KQL queries and ADX clusters to handle large-scale time-series data efficiently
* Implement data retention, indexing, partitioning, and archival strategies for optimized storage and retrieval
* Ensure high availability and fault tolerance in real-time streaming architectures
Visualization & Analytics Enablement:
* Enable real-time insights via Power BI, dashboards, alerts, and anomaly detection
* Collaborate with Data Science and BI teams to support predictive analytics and machine learning on time-series data
* Implement real-time triggers, alerts, and automated responses based on streaming insights
Requirements:
* 3+ years of experience in Microsoft Fabric, Real-Time Analytics (RTA), or Azure Data Explorer (ADX)
* Strong expertise in Kusto Query Language (KQL) and ADX data modeling
* Hands-on experience with event-driven architectures, data streaming, and IoT analytics
* Experience integrating with Event Hubs, IoT Hub, Kafka, or other real-time data sources
* Knowledge of time-series data processing techniques, downsampling, and anomaly detection
* Familiarity with Microsoft Synapse, Power BI, Databricks, or other analytics platforms