Description

Design and manage large-scale data architectures and analytics workloads.

This role is hybrid.

Responsibilities

• Design and develop big data analytics workloads using Apache Spark and Scala.
• Create and optimize data transformation pipelines using Spark or Apache Flink.
• Migrate existing analytics workloads from cloud platforms to open-source Apache Spark.
• Lead and mentor a team of data engineers, analysts, and architects.
• Collaborate with data scientists, business analysts, and developers for seamless integration.

Required Skills

• 5+ years of experience in data architecture or engineering.
• Deep knowledge of Apache Spark, including Spark SQL, Data Frame APIs, and Structured Streaming.
• Proficiency in Scala for big data processing.
• Experience with Apache Flink in both streaming and batching modes.
• Expertise in performance tuning and optimization of Spark jobs.
• Strong understanding of Data Lake, Big Data, ETL processes, and data warehousing.
• Knowledge of lakehouse storage technologies like Delta Lake and Apache Iceberg.
• Experience with AWS and infrastructure running on Kubernetes.
• Expertise in data modeling and optimization for large-scale datasets.

Key Skills

Education

Any Graduate