Big Data Engineer Role
We are seeking a skilled Big Data Engineer to join our team. As a key member of our data engineering group, you will be responsible for designing, developing, and maintaining large-scale data processing systems using cutting-edge technologies.
Your primary focus will be on working with big data processing frameworks such as Apache Spark and Kafka. You will also be involved in the development of data pipelines, ensuring seamless data ingestion, curation, and extraction processes. Additionally, you will be responsible for troubleshooting issues, monitoring system performance, and documenting best practices for future improvements.
* Maintain and develop high-performance data pipelines using Spark, Kafka, and other big data tools.
* Design and implement scalable data storage solutions to meet business requirements.
* Collaborate with cross-functional teams to ensure successful data integration and delivery.
* Develop and maintain data quality metrics, ensuring accurate and reliable data.
* Stay up-to-date with emerging trends and technologies in big data processing and analytics.
Requirements
To succeed in this role, you will need:
* A degree in Computer Science, Software Engineering, or a related field.
* At least 3 years of experience in data engineering, software development, or a related field.
* Strong programming skills in Scala, Python, or Java.
* Expertise in Spark, Kafka, and big data processing concepts.
* Experience with data storage solutions, such as Hadoop HDFS or Amazon S3, and data serialization formats.
* Familiarity with database concepts, data modeling, schema design, and normalization.
* Knowledge of monitoring and logging tools, and workflow management tools, such as Apache Airflow.
Benefits
As a valued member of our team, you can expect:
* A competitive compensation package.
* Opportunities for growth and professional development.
* A collaborative and dynamic work environment.
Why Us
We offer a unique opportunity to work with a talented team of professionals who are passionate about delivering innovative solutions. If you are a motivated and results-driven individual with a passion for big data processing, we encourage you to apply for this exciting opportunity.