Description

You will design and develop big data applications and data warehouse solutions using open source technologies.

Responsibilities

• Design and develop logical and physical data models for big data platforms and RDBMS.
• Build data pipelines using Apache Hive, Apache Spark, Scala, and Apache Kafka.
• Automate workflows using Apache Airflow.
• Lead daily standups, design reviews, and backlog grooming using JIRA.
• Mentor junior engineers and act as the primary point of contact for assigned business domains.

Required Skills

• 8+ years of experience in SQL, Spark, Scala, and Hive.
• 8+ years of experience with ETL processes and data management.
• 5+ years of experience with GCP, including GCS, Dataproc, and BigQuery.
• 4+ years of experience developing distributed data processing platforms using Hadoop, Hive, Scala, or Airflow.
• 2+ years of experience in data modeling using Erwin for data lakes or RDBMS.
• Experience with Python, Java, Scala, Perl, or Shell scripting.
• Experience managing large-scale datasets at the multi-TB or PB scale.
• Background in Scrum/Agile development and test-driven development.
• Bachelor's degree in Computer Science or equivalent experience.

Preferred Skills

• Experience with Gitflow and Atlassian products including BitBucket, JIRA, and Confluence.
• Experience with CI tools such as Bamboo, Jenkins, or TFS.

Key Skills

Education

ANY GRADUATE