Description

You will build and manage large-scale data pipelines and ecosystems within a Big Data environment.

Responsibilities

• Write clean, efficient code in Scala for data processing.
• Design and implement components within the Hadoop ecosystem.
• Manage large data sets and complex data pipelines.
• Utilize workflow scheduling and orchestration tools to automate processes.
• Maintain version control and deployment pipelines using CI/CD and Git.

Required Skills

• 5+ years of experience in Big Data development.
• Proficiency in Scala and Spark.
• Strong experience with SQL, Spark SQL, and Hive QL.
• Hands-on experience with Hadoop and Hive.
• Experience with AWS tools, specifically EMR and EC2.
• Knowledge of CI/CD processes and Git.
• Ability to work with large-scale data orchestration.
• Any Graduate degree.

Preferred Skills

• Experience with Databricks.

Key Skills

Education

Any Graduate