Description

Build and manage data pipelines to support evolving business requirements in Atlanta, GA.

Responsibilities

• Design and implement data pipelines using PySpark and streaming data platforms.
• Manage database storage and retrieval mechanisms specifically for MongoDB.
• Handle all processors and maintain data integrity across streaming and batch environments.
• Collaborate with business stakeholders to translate requirements into technical solutions.
• Guide the team toward integral solution approaches and provide development support.

Required Skills

• 5+ years of experience as a PySpark Developer or in a related data engineering role.
• Proficiency in Scala, Spark, and Python.
• Hands-on experience with AWS cloud and AWS DocumentDB.
• Strong knowledge of Kafka topics and K-SQL.
• Extensive experience with NoSQL databases, specifically MongoDB.
• Deep understanding of Unix, FTP, and file handling.
• Experience working within Agile methodologies using JIRA and Confluence.

Preferred Skills

• Experience with SQL and big data processing techniques.

Key Skills

Education

Bachelor's degree in Computer Science