Description

Design and maintain scalable Big Data architectures and processing pipelines.

This role is remote.

Responsibilities

• Design and implement scalable Big Data architectures using Apache Spark, Hive, and Yarn.
• Develop and optimize Spark data processing pipelines for efficient transformation and analysis.
• Implement Hive queries and data warehouse solutions for structured and semi-structured data.
• Build ETL processes to ingest, transform, and load large volumes of data from diverse sources.
• Troubleshoot and resolve performance bottlenecks in cluster and job execution.

Required Skills

• 6+ years of hands-on experience with Big Data technologies.
• Deep expertise in Apache Spark, Hive, and Yarn.
• Proficiency in Java, Scala, or Python.
• Strong understanding of distributed computing principles and data processing frameworks.
• Experience designing and implementing ETL processes for large-scale datasets.
• Hands-on experience with Sqoop and Big Data ecosystems.
• Proven ability to optimize performance in distributed environments.

Preferred Skills

• Bachelor's degree or equivalent experience in Computer Science, Engineering, or a related field.

Key Skills

Education

Any Graduate