Description

Lead technical teams in building data ingestion and integration pipelines across diverse environments. This role is on-site.

Responsibilities

• Lead technical teams for data ingestion projects moving data from Netezza, Hadoop, and Kafka to AWS EMR, S3, and Hive.
• Design solution approaches and build data pipelines for ingestion into AWS S3 and Hive databases.
• Operationalize data ingestion jobs and applications in the AWS Cloud Data Lake using Apache Airflow.
• Manage AWS services including SageMaker, Apache Airflow, Hue, Athena, Redshift, and EMR cluster spinning via CDK.
• Oversee the AWS platform, release management processes, and change requests for Cloud Data Lake environments.

Required Skills

• 5+ years of experience in big data development.
• Bachelor's degree in Computer Science.
• Proficiency with AWS services (S3, EMR, Redshift, Athena, SageMaker).
• Hands-on experience with Apache Airflow.
• Experience with Kafka and Hadoop environments.
• Knowledge of data integration from Netezza Datawarehouse.
• Experience using CDK for infrastructure processes.
• Background in managing Cloud Data Lake environments.

Key Skills

Education

Bachelor's degree in Computer Science