Description

You will design, develop, and optimize big data pipelines using Apache Spark to enable advanced analytics and data-driven decision-making.

Responsibilities

• Create and optimize data transformation pipelines using Spark or Apache Flink.
• Migrate existing analytics workloads from cloud platforms to open-source Apache Spark infrastructure running on Kubernetes.
• Implement performance optimization techniques for large-scale data processing.
• Ensure efficient data processing, transformation, and storage across distributed systems.

Required Skills

• 6+ years of experience in data engineering or analytics.
• Proficiency in Scala, Python, or Java.
• Experience designing and developing analytics workloads using Apache Spark.
• Hands-on experience with ETL solutions.
• Knowledge of cloud platforms including AWS, Azure, or GCP.
• Bachelor’s Degree in Computer Science, Information Technology, or a related field.

Key Skills

Education

Bachelor’s Degree in Computer Science, Information Technology