Description

You will design, develop, and maintain data processing frameworks and pipelines within an AWS environment.

This role is remote.

Responsibilities

• Design and develop software frameworks using Spark, Hadoop MR, and Kafka.
• Build and maintain data pipelines and data preparation modules.
• Implement streaming solutions and manage the Hadoop ecosystem.
• Develop data processing logic using Java, Scala, or Python.

Required Skills

• 5+ years of experience with Java programming.
• Proficiency in Scala and Python.
• Intermediate knowledge of SQL with AWS Redshift, MySQL, Postgres, or Oracle.
• Hands-on experience with Kafka and streaming technologies.
• Experience with the Hadoop ecosystem and Hadoop MR.
• Experience with Spark.
• Experience with AWS cloud services including S3, Lambda, RDS, and Airflow.

Preferred Skills

• B.S. in Computer Science or a related technical discipline.

Key Skills

Education

B.S. in Computer Science