Description

Build and maintain scalable data processing pipelines and infrastructure in Bengaluru.

This role is on-site.

Responsibilities

• Design and implement scalable data processing pipelines using Apache Spark.
• Develop and optimize Spark jobs for data transformation, aggregation, and analysis.
• Extract, process, and analyze data from various sources within large datasets.
• Implement data integration solutions to connect disparate data sources.
• Monitor and troubleshoot performance issues in Spark jobs and cluster environments.

Required Skills

• 5+ years of experience in data engineering.
• Proficiency with Apache Spark (Java or PySpark) for large-scale processing.
• Strong SQL skills for querying, managing, and manipulating datasets.
• Hands-on experience with Java or PySpark.
• Experience with Azure for data storage, processing, and deployment.
• Ability to collaborate with data scientists and analysts to deliver technical solutions.
• Any Graduate degree.

Preferred Skills

• Experience with large-scale distributed systems.
• Familiarity with data modeling best practices.

Key Skills

Education

Any Graduate