Job Description:
· years' experience working in Data Engineering and Warehousing.
· years' experience integrating data into analytical platforms
· Experience in ingestion technologies (e.g. Sqoop, flume), processing technologies (Spark/Scala) and storage (e.g. HDFS, HBase, Hive)
· Experience in data profiling, source-target mappings, ETL development, SQL optimization, testing and implementation
· Expertise in streaming frameworks (Kafka/Spark Streaming/Storm) essential
· Experience in building Spark Sql and Spark Data Frame API based applications.
· Experience managing structured and unstructured data types