Description

You will design, develop, and maintain scalable data processing solutions and large-scale pipelines.

Responsibilities

• Build and maintain large-scale data processing pipelines using Apache Spark.
• Optimize and tune Spark jobs for performance and scalability.
• Implement data integration solutions to extract, transform, and load data from various sources.
• Ensure data quality and integrity through testing and validation.
• Troubleshoot and resolve issues related to data processing and job execution.

Required Skills

• 5+ years of experience as a Spark Developer or in a similar role.
• Proficiency with Apache Spark for data processing and analytics.
• Strong programming skills in Scala, Java, or Python.
• Experience with big data technologies including Hadoop, Hive, and Kafka.
• Knowledge of SQL and experience working with relational databases.
• Familiarity with data storage solutions such as HDFS or S3.
• Bachelor's Degree in Computer Science, Engineering, or a related field.

Key Skills

Education

Bachelor's Degree