Description

You will build and manage data pipelines and streaming platforms within an AWS environment. You own the technical stack decisions and guide the team through solution approaches. You translate business requirements into technical solutions and ensure project milestones are met.

Responsibilities

• Design and implement data pipelines and streamsets using Kafka.
• Manage database storage and retrieval mechanisms for MongoDB and AWS DocuDB.
• Guide the team through technical stack decisions and integral solution approaches.
• Translate evolving business requirements into technical solutions.
• Provide development support to ensure project milestones are met.

Required Skills

• 5+ years of experience as a PySpark Developer or in a related data engineering role.
• Proficiency in PySpark, Scala, or Python.
• Hands-on experience with AWS cloud services including S3, Athena, Glue, EMR, and VPC.
• Strong expertise in Kafka, including Kafka topics and K-SQL.
• Experience with NoSQL databases, specifically MongoDB and AWS DocuDB.
• Working knowledge of Unix, FTP, and file handling.
• Experience working within Agile methodologies using JIRA and Confluence.
• Background in big data processing tools and streaming data platforms.

Key Skills

Education

Any Graduate