Description

You will design, build, and optimize batch and real-time data pipelines for large-scale processing.

This role is on-site.

Responsibilities

• Design and develop optimized data pipelines for both batch and real-time data processing.
• Implement data warehousing concepts to support large-scale data storage and retrieval.
• Write and optimize complex SQL queries using Hive and PySpark data frames.
• Optimize joins and processing logic to handle massive datasets efficiently.

Required Skills

• 5+ years of experience with Python or Java.
• Deep understanding of Hadoop and Spark architecture and working principles.
• Hands-on experience with SQL, including Hive and PySpark data frames.
• Strong knowledge of data warehousing concepts.
• Proven ability to optimize joins when processing large volumes of data.
• Experience building both batch and real-time data pipelines.

Preferred Skills

• Any Graduate degree.

Key Skills

Education

Any Graduate