Description

You will design, develop, and maintain large-scale data processing pipelines using Apache Spark.

Responsibilities

• Write efficient Spark code to process, transform, and analyze large datasets.
• Optimize Spark jobs for performance, scalability, and resource utilization.
• Integrate Hadoop, Hive, Spring, Hibernate, Kafka, and ETL processes into Spark applications.
• Troubleshoot and resolve issues related to data pipelines and Spark applications.
• Implement data quality and validation processes to ensure accuracy and consistency.

Required Skills

• 10+ years of experience as a Spark Developer or in a similar big data role.
• Strong proficiency in Apache Spark, including Spark SQL, Spark Streaming, and Spark MLlib.
• Proficiency in Scala or Python for Spark development.
• Experience with ETL, data warehousing, and data modeling.
• Solid understanding of distributed computing principles and cluster management.
• Knowledge of Apache Spark, Apache Spark Streaming, Datasets, and ETL processes.
• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

Key Skills

Education

ANY GRADUATE