Description

Design and optimize scalable data pipelines in cloud environments to support marketing analytics and customer data initiatives. Own the infrastructure, orchestration, and warehousing layers while collaborating with cross-functional teams.

Responsibilities

• Design, develop, and orchestrate scalable data workflows and pipelines using Apache Airflow and Apache Spark.
• Build and manage data infrastructure using AWS services including S3, Glue, Athena, Redshift, and EMR.
• Develop modular, maintainable Python code for data engineering and analytics use cases.
• Perform advanced data modeling, transformation, and analysis using SQL, including star and snowflake schemas.
• Implement real-time data streaming pipelines using Kafka and maintain CI/CD pipelines for data applications.

Required Skills

• 7+ years of experience in Big Data Engineering or related data engineering roles.
• Strong programming expertise in Python for data engineering and analytics solutions.
• Advanced SQL skills for data modeling, transformation, and complex data analysis.
• Hands-on experience with Apache Airflow and Apache Spark for workflow orchestration and data processing.
• Strong knowledge of AWS services including S3, Glue, Athena, Redshift, and EMR.
• Experience working with Snowflake and Amazon Redshift data warehouses.
• Experience with real-time streaming technologies such as Kafka.
• Experience implementing CI/CD pipelines for data engineering solutions.
• Familiarity with Agile methodologies such as Scrum, Kanban, or SAFe.

Key Skills

Education

Any Graduate