Description

Design and develop distributed data processing platforms for multi-TB/PB scale datasets.

This role is on-site.

Responsibilities

• Design and develop logical and physical data models for big data platforms.
• Build data pipelines using Apache Hive, Apache Spark, Scala, and Apache Kafka.
• Automate workflows using Apache Airflow and maintain existing systems.
• Lead daily standups, design reviews, and backlog grooming using JIRA.
• Mentor junior engineers and act as the primary point of contact for assigned business domains.

Required Skills

• 8+ years of experience developing data warehouse solutions and data products.
• 4+ years of experience with Hadoop, Hive, Scala, Airflow, or workflow orchestration.
• 4+ years of experience with GCP, GCS Dataproc, and BigQuery.
• 2+ years of experience in schema design and modeling using Erwin for data lakes or RDBMS.
• Proficiency in Python, Java, and Scala.
• Experience with scripting languages including Perl and Shell.
• Background in Scrum/Agile methodologies and test-driven development.

Preferred Skills

• Experience with Gitflow and Atlassian products including BitBucket and Confluence.
• Experience with CI tools such as Bamboo, Jenkins, or TFS.

Key Skills

Education

Any Graduate