Description

You will own core responsibilities in building and optimizing data pipelines within a Big Data environment.

Responsibilities

• Develop and maintain data integration projects utilizing Big Data technologies.
• Build Kafka-based data ingestion and retrieval programs.
• Tune Hadoop, Spark, and Hive parameters for performance optimization.
• Ensure high code quality through rigorous automated testing and engineering best practices.

Required Skills

• 7+ years of strong Java programming experience.
• 3+ years of experience with Hadoop MapReduce, Sqoop, Oozie, Hive, and Spark.
• Solid experience with Kafka, specifically for data ingestion/retrieval.
• 2+ years of AWS experience, including Lambda, S3, Redshift, and Glue services.
• Strong SQL query writing and data analysis skills.
• Experience migrating workloads to AWS.
• Good shell scripting experience.
• Ability to write reusable code components.

Key Skills

Education

Any Graduate