Description

Design and implement distributed data processing solutions within a big data ecosystem. You will own end-to-end ETL and data pipelines, contribute to architectural decisions, and solve high-level technical problems.

Responsibilities

• Build and maintain end-to-end ETL and data pipelines within Hadoop environments.
• Develop core frameworks and templates to standardize team implementation and execution.
• Debug code, manage error handling, and ensure timely, high-quality deliveries.
• Collaborate with geographically distributed teams and interact with customers daily.

Required Skills

• 5 to 8 years of experience in data engineering within open-source, data-intensive, and distributed environments.
• Minimum 5 years of hands-on experience with Spark, Hive, HBase, and Hadoop.
• Proficiency with MapReduce, HDFS, YARN, and Spark Streaming.
• Strong programming skills in Python and Scala.
• Experience with SQL, T-SQL, and ETL pipeline development.
• Working knowledge of Unix, Linux, and GitHub.
• Experience using Jenkins for CI/CD and debugging workflows.
• Practical experience with Google Cloud Platform.
• Ability to work within Agile methodologies.

Key Skills

Education

Any Gradute