You will own the development, deployment, and scaling of big data and platform data management infrastructure.
Responsibilities
- Orchestrate, deploy, and maintain cloud or on-premise infrastructure for big data and platform data management (Relational and NoSQL).
- Design and build Ab-Initio data graphs and pipelines to extract data from various sources like databases, flat files, and message queues.
- Transform extracted data to create a consumable data layer for application use.
- Support data pipelines by fixing bugs and implementing enhancements.
- Document technical designs and operational runbooks.
Required Skills
- 10+ years of IT experience, predominantly in Data Integration/Data Warehouse.
- 5+ years of ETL Design and Development experience using Ab-Initio.
- Working knowledge of HDFS, Hive, and Impala.
- Experience integrating Ab-Initio with AWS S3 and Redshift or other AWS database services.
- Strong understanding of SQL and ability to write performant queries.
- Experience with Unix/Linux shell scripting.
- Knowledge of Agile Development practices.
- Ability to unit test code thoroughly and troubleshoot production issues.
- Familiarity with OLTP and OLAP data models.