About the Role
You will be joining our organization as a Data Infrastructure Engineer, playing a key role in architecting and maintaining a hybrid on-premises and cloud-based data platform.
Key Responsibilities:
* Design and Build Scalable Data Infrastructure: Develop and maintain high-performance data infrastructure across on-premises and cloud environments.
* Develop Real-Time and Batch-Oriented Data Pipelines: Design and implement real-time and batch-oriented data pipelines for market data ingestion.
* Integrate and Manage Data: Integrate and manage data from internal systems and external vendors with a focus on reliability and quality.
* Implement Best Practices: Implement best practices around data governance, access control, monitoring, and observability.
* Collaborate with Stakeholders: Collaborate with traders, researchers, and developers to align data systems with business needs.
* Automate Infrastructure Provisioning: Automate infrastructure provisioning and deployments using modern DevOps and IaC tools.
* Assist and Mentor Others: Assist and mentor others working on data pipelines and orchestration.
Requirements:
* Degree and Experience: Degree in Maths, Engineering, Computer Science, or comparable experience.
* Technical Skills: Strong understanding of distributed systems, data storage (e.g. S3, columnar stores), and streaming technologies (e.g. Kafka).
* Programming Languages: Proficiency in Python, Go, or Java.
* Cloud Platforms: Cloud platform experience (AWS, GCP).
* SQL and Data Modelling: SQL and data modelling skills.
* Kubernetes and Docker: Experience with Kubernetes, Docker, and infrastructure as code.
* Orchestration Tools: Familiarity with orchestration tools like Airflow or Dagster.
* Communication and Learning: Strong communication skills and fluency in English. A passion for continuous learning.