Description

Job Description

Responsibilities:

• Develop, test, and maintain big data solutions using Apache Spark.
• Collaborate with data engineers, data scientists, and other stakeholders to understand data requirements and translate them into scalable data processing workflows.
• Optimize Spark jobs for performance and scalability.
• Write high-quality, clean, and maintainable code.
• Monitor and troubleshoot production data pipelines.
• Implement data quality checks and ensure data integrity.
• Stay up to date with the latest trends and advancements in big data technologies.

Requirements:

• Proven experience as a Spark Developer or in a similar role.
• Strong knowledge of Apache Spark and its ecosystem.
• Experience with big data processing frameworks such as Hadoop.
• Proficiency in programming languages such as Scala, Java, or Python.
• Experience with data storage and retrieval technologies such as HDFS, HBase, Cassandra, or similar.
• Familiarity with data integration tools and ETL processes.
• Understanding of distributed computing principles.

Skills:

• Strong problem-solving and analytical skills.
• Excellent communication and collaboration abilities.
• Ability to work in a fast-paced, agile environment.
• Knowledge of performance tuning and optimization techniques for Spark.
• Experience with cloud platforms (AWS, Azure, Google Cloud) is a plus.
• Familiarity with streaming technologies such as Apache Kafka, Flink, or similar is a plus

Key Skills

Education

Any Graduate