Description

You will build and maintain data pipelines and manage infrastructure for large-scale data processing.

This role is on-site.

Responsibilities

• Develop data processing solutions using Spark or PySpark for TB-scale data.
• Manage AWS infrastructure and configuration using Terraform.
• Build and maintain CI/CD and data pipelines using Python.
• Process large volumes of data using tools like Snowflake and Redshift.
• Work with messaging and queue services (SNS, SQS) within AWS.

Required Skills

• 5+ years of professional experience.
• Expert proficiency in Python.
• Strong experience with Spark or PySpark.
• Deep knowledge of AWS services, specifically S3 and EMR.
• Proficiency in Infrastructure as Code using Terraform.
• Experience with batch data processing at TB scale.
• Familiarity with Java.
• Experience with Snowflake and Redshift.
• Working knowledge of SQL.

Key Skills

Education

Any Graduate