Description

You will lead the design, development, and maintenance of data processing pipelines.

Responsibilities

• Design, develop, and maintain data processing pipelines using Apache Spark.
• Optimize Spark jobs for performance and scalability in distributed environments.
• Perform ETL operations on large datasets.
• Collaborate with data scientists, analysts, and engineers to deliver data solutions.
• Troubleshoot and resolve data processing and performance issues.

Required Skills

• Proficiency in Apache Spark, Spark Core, and Spark SQL.
• Strong programming skills in Scala, Python, or Java.
• Experience with ETL processes.
• Familiarity with the Hadoop ecosystem.
• Solid understanding of data structures and algorithms.
• Experience with distributed computing concepts.
• Experience working with Data Scientists and Analysts.

Key Skills

Education

Any Gradute