Job Overview
We are seeking a talented Data Platform Expert to join our team focused on building scalable data systems for high-frequency telemetry data.
Key Responsibilities
* Design and develop robust, scalable components for ingesting, processing, and persisting high-frequency telemetry data using Apache Spark, Delta Lake, and PostgreSQL.
* Collaborate with data scientists to host, orchestrate and optimize workloads in Python, Scala, and Java.
* Build components using technologies like Redis, MQTT, and Ceph S3-compatible object stores.
* Drive modernization efforts including containerization and deployment on Kubernetes.
* Evaluate and integrate emerging technologies to optimize performance and scalability.
* Contribute to architectural decisions involving event sourcing, CQRS, and hybrid cloud deployments.
Requirements
* Extensive experience in backend development with languages such as Java, Scala, and Python.
* Proven track record working in teams to develop large, complex applications.
* Deep understanding of streaming and batch data processing, ideally with Apache Spark or similar.
* Experience with containerization (Docker) and orchestration (Kubernetes).
* Familiarity with data lake/lakehouse architectures, especially Delta Lake.
* Strong knowledge of message brokers (MQTT, Kafka) and caching systems (Redis).
* Comfortable working across multiple languages (Java, Python, Scala).
* Experience in designing systems for scalability, multi-tenancy, and hybrid deployments.
Why Work with Us?
* Be part of a forward-thinking team driving innovation in data-centric systems.
* Influence architectural direction and technology choices.
* Collaborate with experts in autonomy, data science, and distributed systems.
Working Conditions
The ideal candidate will have excellent communication skills, be able to work collaboratively in a team environment, and demonstrate a strong passion for building scalable data systems.
Skills Assessment
* Proficiency in Java, Scala, or Python programming languages.
* Experience with containerization and orchestration tools.
* Understanding of data storage and processing technologies.
* Knowledge of data visualization and business intelligence tools.
* Ability to design and implement scalable data systems.