Description

Build and manage large-scale data processing systems using Apache Spark.

Responsibilities

• Design and implement scalable data processing pipelines using Apache Spark.
• Develop and optimize Spark jobs for data transformation, aggregation, and analysis.
• Extract, process, and analyze data from various sources within large datasets.
• Implement data integration solutions to connect disparate data sources.
• Monitor and troubleshoot performance issues in Spark jobs and cluster environments.

Required Skills

• 5+ years of experience in data engineering.
• Proficiency with Apache Spark using Java or PySpark.
• Strong SQL skills for querying, managing, and manipulating datasets.
• Experience with Azure cloud platforms for data storage and processing.
• Hands-on experience with large-scale data processing.
• Ability to collaborate with data scientists and analysts to deliver technical solutions.
• Any Graduate degree.

Key Skills

Education

Any Graduate