Opportunity to work for this dynamic, fast paced FMCG company as a Data Engineer where you will be responsible for the preparation, development, testing and maintenance of data engineering which includes building the data pipelines, data warehouse integration, analysis, profiling, data development and production infrastructure.
Your brand-new role will also see you responsible for building data ingestion and data transformation infrastructure while also playing an integral role in the building, managing, supporting enterprise data platform that empower the business stakeholders to drive data value and insights. and ultimately enterprise data operations
Key activities
1. Translate strategic requirements to ensure effective solutions meet business requirements
2. Perform systems and applications performance characterization and trade-off studies through analysis and simulation
3. Embed business intelligence best practices dimensional modelling, ETL pipeline, large scale distributed ETL pipelines) to enable enterprise data insights and large-scale machine learning.
4. Create and maintain analytics data pipelines that generate data + insight to power business decision making
5. Develop and operate modern data architecture approaches to meet key business objectives and provide end-to-end data solutions
6. Create data models and speak to the trade-offs of different modelling approaches
7. Design, model and maintain data structures for analytics purposes
8. Maintain and configure data engineering and ETL/ELT tools
9. Improve the data availability by acting as a liaison between analytics teams and source systems
10. Collect, blend, and transform data using ETL tools, types of databases, and code development (SQL, Spark, AWS Glue etc) for batch and streaming use cases
Skills and experience
11. Minimum 5-7+ years' experience in a similar data engineering position
12. Extensive knowledge of BI concepts ETL, dimensional modelling, data warehouse design, dashboarding)
13. Strong experience with cloud-based data warehousing, data architecture and data lake concepts
14. Knowledge of query languages SQL, Spark), database design, optimising queries, internals knowledge of query planning
15. Experience in managing data pipelines with tools and packages such as Talend, Informatica, Airflow
16. Experience with delivering on AWS or Azure cloud environments
17. Proficient in AWS data engineering technical stack, AWS Glue, SQL, Python, dbt cloud or core, AWS S3, Kinesis,RDS, Dynamo DB, Athena, Linux, Unix, windows shell scripting
18. Experience in database technologies such as snowflake, AWS redshift, MS SQL Server, Oracle, Teradata etc
19. Experience in DevOps tech stack in supporting CI/CD pipelines including documentation