Description

You will design and develop big data applications and distributed processing platforms on Google Cloud Platform.

Responsibilities

• Design and build big data applications using open-source technologies.
• Develop logical and physical data models for data lakes and platforms.
• Automate workflows with Apache Airflow and build pipelines using Spark, Hive, and Kafka.
• Lead design reviews, manage JIRA backlogs, and mentor junior engineers.
• Maintain existing systems and participate in rotational on-call support.

Required Skills

• 8+ years developing data warehouse solutions and data products.
• 4+ years with Hadoop, Hive, Scala, or Airflow.
• 4+ years with GCP, GCS Dataproc, and BigQuery.
• 2+ years in schema design using Erwin for data lakes or RDBMS.
• Proficiency in Python, Java, and Scala.
• Experience with Perl and Shell scripting.
• Proven experience managing multi-TB or PB-scale datasets.
• Familiarity with Agile/Scrum and test-driven development.
• Bachelor's Degree in Computer Science or equivalent experience.

Preferred Skills

• Experience with Gitflow and Atlassian products (BitBucket, Confluence).
• Experience with CI tools like Bamboo, Jenkins, or TFS.

Key Skills

Education

Any Graduate