Description

You will manage Spark clusters and platform engineering tasks within a production environment.

Responsibilities

• Manage Spark clusters and connections to ensure stable production operations.
• Monitor system health and performance using Splunk and Elastic.
• Troubleshoot and debug issues within production environments by analyzing logs.
• Manage data pipelines and migrate platforms to production.
• Develop technical solutions for existing infrastructure challenges.

Required Skills

• 4+ years of experience in platform engineering or cluster management.
• Direct experience with Spark administration.
• Proven ability to manage and troubleshoot Hadoop environments.
• Experience monitoring production systems with Splunk or Elastic.
• Strong debugging skills and log analysis capabilities.
• Experience managing production-level data pipelines.
• Any graduate degree.

Preferred Skills

• Experience working with Kafka in combination with Spark.
• Working knowledge of Kafka.

Key Skills

Education

Any gradudate