AWS Marketing Team is hiring a Sr. Big Data Engineer to build a flexible and scalable data brokerage (ETL and Data Integration) service which is capable of (i) efficiently bringing data from multiple data sources, (ii) transform and process large data-sets in real time, and (iii) supports real-time data sync data between two heterogeneous data sources. This platform will be used by multiple teams within AWS Marketing like Central Business Intelligence team, Data Science team, Marketing Automation team for their data processing and integration needs. Role and Responsibilities
- Build a long-term ETL Infrastructure roadmap, perform POC (proof of concepts), choose the right tools and technologies to be used.
- Build a platform to run micro ETL batches to process data-sets in near real time.
- Move existing ETL jobs from traditional data warehouse processing to the big data processing platform, ensure that the jobs are designed to scale.
- Setup and maintain Data Lake for AWS Marketing team.