Job Role Overview
As a Hadoop Developer, you will play a key role in designing and implementing large-scale data processing systems.
Key Responsibilities:
1. Design and implement Big Data Platform solutions using Cloudera Hadoop platform with minimum 2 years of experience.
2. Develop scalable data pipeline tools & utilities: MapReduce, PySpark, Kafka, Sqoop, Impala, Hive etc., with expertise in Hadoop ecosystem.
3. Collaborate on traditional Enterprise Data Warehouse (EDW) design & development using Informatica or Talend, leveraging your skills in data modeling.
4. Contribute to overall architecture and design, working independently as needed.
5. Write efficient Shell scripts in Linux Platform for automation purposes, focusing on code quality.
6. Apply knowledge of API management concepts and design principles, ensuring secure integration.
7. Automate CI/CD deployment in Bigdata Platform, streamlining the process.
8. Manage Metadata using Informatica Metadata Manager or Collibra, maintaining data accuracy.