Description

You will build and manage data pipelines and streaming platforms within an AWS environment, owning technical stack decisions and guiding the team through solution approaches.

Responsibilities

• Design and implement data pipelines and streamsets using Kafka.
• Manage database storage and retrieval mechanisms for MongoDB and AWS DocuDB.
• Translate evolving business requirements into technical solutions.
• Guide the team through technical stack decisions and integral solution approaches.
• Provide development support to ensure project milestones are met.

Required Skills

• 5+ years of experience as a PySpark Developer or in a related data engineering role.
• Proficiency in PySpark, Scala, or Python.
• Hands-on experience with AWS cloud services including S3, Athena, Glue, EMR, and VPC.
• Strong expertise in Kafka, including Kafka topics and K-SQL.
• Experience with NoSQL databases, specifically MongoDB and AWS DocuDB.
• Working knowledge of Unix, FTP, and file handling.
• Experience working within Agile methodologies using JIRA and Confluence.

Key Skills

Education

Any Graduate