Description

You will lead the development and optimization of data pipelines within an AWS environment.

Responsibilities

• Build and maintain ETL pipelines using Apache Kafka, Spark, and Airflow.
• Implement data pipelines utilizing cloud-native AWS services.
• Optimize data processes using AWS Glue, Redshift, Lambda, and API Gateway.
• Monitor system health and performance via CloudWatch.
• Collaborate with stakeholders, data architects, and data scientists to improve data quality.

Required Skills

• 5+ years of experience in big data technologies.
• Proficiency in Python, Scala, and SQL.
• Hands-on experience with Apache Kafka and Spark.
• Experience with Airflow for workflow orchestration.
• Direct experience with AWS services: Redshift, Glue, Lambda, API Gateway, and CloudWatch.
• Bachelor's degree in Computer Science.

Key Skills

Education

Bachelor's degree in Computer Science